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Introduction 

Funding  was  provided  for  a  conference  on  Image-Guided  Procedures  and  Surgical 
Simulation  at  the  2001  SPIE  Medical  Imaging  meeting.  Funds  were  used  to  provide  travel  re¬ 
imbursement  for  speakers,  for  the  cost  of  putting  on  a  hands-on  workshop  and  for  creating  a 
deliverable:  a  snapshot  of  the  state  of  the  art  in  image-guided  procedures  in  2001 . 


Body: 

The  organizers  of  the  2001  SPIE  Medical  Imaging  meeting  decided  to  “retask”  the  Image 
Display  Conference  to  have  a  separate  Image-Guided  Procedures  and  Surgical  Simulation  tract. 
Funds  from  the  USMRMC  and  from  the  Whitaker  foundation  allowed  us  to  bring  in  some  of  the 
top  researchers  in  this  field  as  well  as  up  and  coming  young  scholars.  A  short  course  on  surgical 
simulation  and  a  hands-on  workshop  were  developed. 

We  had  hoped  to  get  at  least  20  abstracts  so  we  could  fill  one  day  of  the  meeting,  we  received  more 
t-han  90!  Since  image-guided  procedure  systems  are  kinetic,  they  are  often  undersold  by  slide 
displays  so  it  was  decided  that  we  would  put  on  a  workshop  where  systems  could  be  brought  in  for  a 
hands-on  demonstration.  Again,  our  expectations  were  modest,  we  were  up  against  two  other  high- 
profile  workshops  and  we  hoped  for  30  to  60  attendees,  we  got  more  than  300!  So  clearly  this  is  an 
area  of  some  major  interest. 


KEY  RESEARCH  ACCOMPLISHMENTS: 

The  Image-Guided  Procedures  and  Surgical  Simulation  tract  of  the  SPIE  meeting  proved  to  be  a 
great  success.  Over  60  papers  and  posters  were  presented  and  the  workshop  generated 
significant  interest  in  the  field. 


REPORTABLE  OUTCOMES: 

In  the  appendix  is  a  document  of  providing  a  snapshot  report  of  the  state  of  the  art  in  Image- 
Guided  Procedures  and  Surgical  Simulations  as  of  2001. 


CONCLUSIONS: 

The  Image-Guided  Procedures  and  Surgical  Simulation  Tract  for  the  meeting  was  a  huge 
success.  The  hand-on  workshop  generated  significant  interest  beyond  that  of  the  papers. 


REFERENCES:  NA 

APPENDICES:  Copies  of  the  Snapshot  Document  are  enclosed. 
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Introduction 

While  medical  images  have  been  used  to  assist  surgery  for  almost  as  long  as  there  have  been  medical 
images,  it  is  only  with  the  rise  of  image-assisted  stereotaxy  in  the  late  1940s  that  the  images  were 
used  quantitatively.  With  the  development  of  computed  tomography  in  the  1970s,  medical  images 
as  numerical  data  sets  emerged.  In  the  mid  1980’s  a  number  of  researchers  world  wide  realized  that 
these  image  sets  had  value  beyond  their  diagnostic  capabilities  and  with  the  development  of  small 
inexpensive  computers  set  about  developing  system  to  exploit  the  localization  and  guidance 
potential  in  such  sets.  So  here  in  2001  we  are  dealing  with  a  technology  which  can  be  considered  15 
years  old  at  the  most.  The  field  has  gone  through  several  steps:  First,  researchers  developed  systems 
which  determined  and  demonstrated  what  could  be  done.  Then  work  moved  forward  on  the 
development  of  metrics  for  assessing  what  had  been  done.  As  we  begin  to  build  up  a  body  of 
knowledge  about  the  guidance  of  therapuetic  procedures  we  can  begin  to  ask  the  question:  what 
should  be  done?  Image-guidance  has  proven  to  be  enabling,  that  is  allowing  surgeons,  interventional 
radiologists  and  radiation  therapists  the  confidence  to  guide  treatment  to  previously  “untreatable” 
locations.  It  has  also  proven  to  be  safe,  actually  reducing  OR  time  and  the  associated  morbidity 
relative  to  other  approaches  to  the  same  therapy.  What  is  only  now  being  assembled  is  the  data  on 
the  efficacy  of  what  we  now  call  Image-Guided  Procedures  (IGP). 

While  we  begin  to  get  data  on  the  long-term  outcome  of  some  of  the  clinical  studies,  system 
development,  refinement  and  new  procedures  continue  to  emerge.  The  vast  majority  of  work  in  the 
field  has  been  in  neurosurgical  applications  as  an  outgrowth  of  stereotaxy,  but  as  analyzed  as  a 
health-care  issue,  neurosurgery  is  a  thankfully  small  problem.  We  are  beginning  to  see  applications 
outside  of  brain  surgery  and  the  required,  accompanying  development  of  technology  asociated  whith 
those  applications.  Beyond  that  we  are  seeing  true  integration  of  multi-disciplinary  teams  composed 
of  surgeons,  radiation  therapists,  engineers,  physcists  and  computer  scientists.  Engineering  graduate 
students  working  with  surgical  residents,  graduate  students  being  funded  by  clinical  departments  and 
residents  doing  research  in  engineering  laboratories  all  bode  for  explosive  future  growtrh  in  the  field. 

It  is  that  growth  and  the  newness  of  the  ideas,  methods  and  principles  of  IGP  which  make  this 
merely  snapshot  of  the  field.  We  are  in  the  first  generation  of  systems  and  as  so  often  been  the  case, 
arrival  at  generation  one  merely  points  up  the  places  where  approximation  and  ignorance  of  the 
reality  occurs.  For  the  most  part,  IGP  systems  treat  tissue  as  if  it  is  rigid  and  monolitihic  in 
construction  and  function.  Those  are  laughable  approximations  of  reality,  but  were  necessary  to 
make  to  get  IGP  systems  up  and  running.  Now  that  we  have  high-accuracy  localization,  more 
sophisticated  assesment  of  registration  quality  and  improved  segmentation  algorithms  we  can  make 
the  measurements  necessary  to  allow  for  tissue  motion  and  deformation,  for  dealing  with  multi¬ 
compartment  structures  and  for  using  functional  information  as  well  as  anatomic  to  guide  the 
delivery  of  therapy.  Beyond  that,  we  are  beginning  to  think  about  the  role  of  IGP  in  improving  the 
specificity  of  therapy  delivery,  not  only  treating  the  disease  but  actively  avoiding  damage  to  healthy 
tissue.  We  are  also  seeing  surgeons,  interventional  radiologist  and  radiation  oncologists  being  trained 
from  the  start  with  a  knowledge  of  the  existence  and  principles  of  IGT.  They  are  proving  to  think 
about  treatment  planning  and  delivery  in  ways  which  much  different  from  their  precessors. 

In  2000  Kevin  Cleary  and  I  were  tasked  with  job  of  creating  an  Image-Guided  Procedures  meeting 
within  the  confines  of  the  SPIE  2001  Medical  Imaging  meeting  (San  Diego,  CA,  Feb  17-22).  While 
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the  SPIE  meeting  had  had  IGP  papers  in  the  past,  this  was  the  first  attempt  to  focus  a  section  of  the 
meeting  on  that  particular  topic.  We  had  hoped  to  get  at  least  20  abstracts  so  we  could  fill  one  day 
of  the  meeting,  we  received  more  than  90!  Since  image-guided  procedure  systems  are  kinetic,  they 
are  often  undersold  by  slide  displays  so  it  was  decided  that  we  would  put  on  a  workshop  where 
systems  could  be  brought  in  for  a  hands-on  demonstration.  Again,  our  expectations  were  modest, 
we  were  up  against  two  other  high-profile  workshops  and  we  hoped  for  30  to  60  attendees,  we  got 
more  than  300! 

So  clearly  this  is  an  area  of  some  major  interest. 

The  papers  composing  this  manuscript  begin  with  a  modified  version  of  a  review  article  that  I  wrote 
to  lay  the  foundation  for  IGP.  Following  that  Dr.  Terrance  Peters  has  provided  his  keynote  address 
on  the  state  of  the  art  in  IGP.  The  next  article  is  a  review  of  Surgical  Simulation,  an  area  of  explosive 
growth  that  uses  a  number  of  the  principles  of  IGP  but  adds  layers  of  tissue  modelling  and  haptic 
interfaces.  Finally,  Dr.  Kevin  Cleary  contributes  a  paper  on  surgical  robotics. 

As  I  said  earlier,  the  field  of  Image-Guided  Procedures  is  in  its  infancy.  This  document  is  to  serve  as 
a  snapshot  of  the  field  in  2001. 


Robert  L.  Galloway  Jr. 
July  2001 
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Background  on  the  development  of  image-guided  procedures 

Robert  L.  Galloway  Jr.  PhD 
Dept.  Of  Biomedical  Engineering 
Dept,  of  Neurological  Surgery 
Vanderbilt  University 

MOTIVATION  FOR  IMAGE  GUIDANCE 

One  of  the  most  fundamental  forces  in  the  development  of  surgery  and  other  forms  of 
directed  therapy  is  the  need  to  increase  the  information  available  to  physicians  and  to  place 
that  information  in  both  spatial  and  temporal  contexts.  From  such  perspectives,  the 
development  of  surgery,  radiation  therapy,  and  now  even  chemotherapy  can  be  followed. 

In  each  field  there  is  a  movement  to  finer  and  finer  preoperative  determinations  of 
pathology  locations  coupled  with  intraprocedural  determinations  of  therapeutic  positions. 
This  process  can  be  seen  as  progression  from  providing  therapy  to  a  general  patient  to 
treating  a  specific  one. 

In  all  therapies,  what  is  needed  for  successful  outcomes  is  knowledge  of  the  spatial 
location  and  extent  of  a  problem.  To  use  surgery  as  an  example,  with  the  advancement  of 
surgical  techniques  came  the  need  for  a  surgeon  to  exacdy  define  the  location  of  his  or  her 
instruments.  Simple  visual  inspection  gave  way  to  magnifying  loupes,  loupes  to  operating 
microscopes  and  endoscopes.  This  relationship  between  knowing  the  spatial  location  and 
extent  of  a  problem  and  performing  the  surgery  is  so  intuitive  that  only  8  days  after  the 
announcement  of  Roentgen's  discovery  of  X  rays,  the  first  image-guided  surgery  was 
performed.  On  January  13,  1896,  a  woman  in  Birmingham  (UK)  buried  a  needle  in  her 
hand.  A  radiograph  was  made  of  her  hand,  which  she  took  to  casualty  surgeon  JH  Clayton 
to  help  him  in  his  surgical  removal  of  the  needle  (1).  Here  the  trauma  was  known,  the  shape 
and  size  of  the  surgical  target  exactly  defined.  The  radiograph  reduced  the  uncertainty  of  the 
location  of  the  needle  relative  to  other  anatomic  structures  in  that  patient.  However, 
because  simple  radiographs  compress  three-dimensional  spatial  relations  into  two- 
dimensional  images,  there  remained  a  degree  of  positional  uncertainty  relative  to  the 
thickness  of  the  patient's  hand.  In  addition,  a  radiograph  shows  only  the  bony  anatomy 
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clearly;  therefore,  the  surgeon’s  knowledge  of  the  location  of  other  structures,  such  as 
tendons  and  vessels,  was  still  dependent  on  his  or  her  understanding  of  general  anatomy. 

This  same  progression  can  be  seen  in  other  therapies  as  well.  Simple  external  beam, 
quadrant-directed  radiotherapy  has  been  enhanced  by  stereotactic  radiosurgery  and  therapy; 
brachytherapy  delivers  radiation  from  the  inside  of  the  tissue  outward;  and  direct-injection, 
controlled-release  chemotherapy  is  replacing  systemic  chemotherapy  in  certain  applications. 

As  imaging  has  progressed  from  simple,  static  X  rays  to  include  tomograms  and  dynamic 
imaging,  the  application  of  those  images  to  therapy  has  moved  with  it.  As  image  sets  became 
three  dimensional  in  space  and  time,  the  technological  complexity  of  putting  them  to  use 
increased  dramatically.  This  necessitated  the  creation  of  physician-engineer  teams  to  develop 
devices  and  techniques,  a  collaboration  that  has  grown  to  become  the  field  of  image-guided 
procedures  (IGPs). 

Because  preinterventional  images  exist  for  the  vast  majority  of  therapeutic  procedures, 
what  constitutes  an  IGP?  One  definition  is  that  in  an  IGP,  images  are  used  quantitatively.  . 
That  is,  their  spatial  parameters  carry  equal  or  greater  weight  than  their  gray-scale  parameters. 
Thus,  the  mere  presence  of  tomographic  images  in  an  operating  room,  or  the  use  of 
intraoperative  imaging  to  visually  align  tool  and  target,  does  not,  by  this  definition,  create  an 
IGP. 

HISTORY 

Approximately  10  years  after  the  surgery  by  Clayton  (1),  Horsley  &  Clarke  (2)  presented  a 
device  that  embodied  two  ideas  fundamental  to  surgical  guidance:  image  space  and  physical 
space.  For  their  process,  known  as  stereotaxy,  to  work,  both  the  image  space  and  the  physical 
space  must  in  some  way  be  registered,  i.e.  the  transformational  relationship  between  two 
three-dimensional  spaces  must  be  determined.  In  this  case  the  image-space  map  consisted  of 
anatomic  drawings  and  sectioned  samples,  but  it  inaugurated  the  concept  that  direct  surgical 
visualization  was  no  longer  necessary  for  the  accurate  placement  and  guidance  of  medical 
devices.  Their  concept  for  charting  the  physical  space  was  an  extracranial  device,  or  frame. 
Horsley  &  Clarke  accomplished  this  by  mounting  on  a  subject  a  frame  based  on  intrinsic 
reference  points,  anatomic  landmarks  such  as  the  external  auditory  canals  and  the  inferior 
orbital  rims.  Based  on  these  landmarks,  the  frame  could  be  identically  mounted  on  every 
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subject  and  the  image-space  maps  could  then  be  used  to  place  electrodes,  make  lesions,  and 
take  tissue  samples.  There  is  an  implied  relationship  here:  that  intracranial  structures  of  the 
brain  have  a  stationary  relationship  to  extracranial  anatomic  landmarks  across  the  subject 
population.  That  is,  the  general  case  sufficiently  defines  the  specific.  However,  the 
fundamental  problem  in  using  this  technique  on  humans  was  that  the  map  between  external 
anatomic  landmarks  and  intracerebral  locations  proved  not  to  be  stationary  across  human 
beings. 

Figure  1:  The 
Horsely  and  Clark 
Stereotactic  Frame. 

Thus,  for  accurate 
guidance, 
generalized  atlases 
and  representative 
tissue  sections  alone 
were  inadequate. 

Although 
stereotactic 
techniques  have 
been  applied  to  a 
number  of  different 
surgical  areas  (3—6), 

stereotaxy  has  found  its  most  common  application  in  neurosurgery  because  of  the  singular 
structure  of  the  cranium.  The  need  to  operate  on  this  structure,  unique  in  that  it  is  a  solid 
organ  almost  entirely  encompassed  by  bone,  leads  to  two  major  constraints.  The  first  is 
with  the  protective  nature  of  the  skull.  Surgeons  prefer  to  make  the  minimum  opening 
required  to  perform  the  surgery  but  must  make  a  large  enough  opening  to  guarantee  access 
to  the  entire  site.  Second,  there  is  the  functional  importance  of  all  the  tissue  between  the 
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surface  of  the  brain  and  the  site  of  surgical  interest.  These  surgical  constraints  have  led  to  a 
desire  to  develop  techniques  for  accurate  location  of  positions  within  the  brain  prior  to  the 
opening  of  the  skull. 

The  term  site  of  surgical  interest  is  meant  to  span  all  the  applications  of  stereotaxy.  The 
first,  and  still  most-often  used,  application  of  stereotaxy  was  for  the  placement  of  electrodes 
to  known  physical  locations  for  the  mapping  of  the  brain's  electrical  activity  and  functional 
response  (7-12).  The  mapping  process  was  followed  by  the  development  of  surgery  that 
attempts  to  change  the  function  of  the  brain  in  some  fashion.  These  procedures,  collectively 
known  as  functional  neurosurgery,  differ  from  other  forms  of  stereotactic  neurosurgery  in 
that  there  is  not  always  a  visible  focus  for  resection,  and  the  surgery  is  not  to  remove  some 
lesion  or  vascular  abnormality  but  to  attempt  to  change  some  undesirable  neurologic 
function.  The  surgical  point  is  determined  by  observation  of  the  local  area's  function, 
generally  through  electrical  monitoring,  and  the  correlation  with  known  functional  locations 
or  anatomic  landmarks. 

In  1947,  Spiegel  &  Wycis  (13)  published  a  paper  describing  their  device  for  human 
stereotaxy.  This  apparatus  did  not  use  skull  landmarks  for  the  location  of  soft-tissue  targets. 
Rather,  it  extracted  information  from  pneumoencephalograms.  By  providing  visual 
information  as  to  the  relationship  between  the  stereotactic  apparatus  and  soft-tissue 
landmarks,  such  as  the  foramen  of  Monroe  and  the  pineal  gland,  electrodes  could  be  guided 
with  much  greater  certainty.  By  taking  care  to  preserve  well-defined  geometries  in  their 
imaging,  each  patient's  films  became  a  specific  atlas  to  guide  electrode  placement.  .  In 
addition,  it  is  an  example  of  the  use  of  image  information  in  a  quantitative  way.  Arguably, 
the  concept  of  using  pneumoencephalograms  and  ventriculography  as  a  mapping  of  the 
topology  of  the  brain  did  not  occur  to  Spiegel  &  Wycis  exclusively.  During  the  next  few 
years,  several  major  stereotactic  systems  were  initiated.  This  included  the  first  Leksell  (14) 
and  Riechert  &  Wolff  (15)  (later  Riechert-Mundinger)  systems.  In  France,  Talairach  et  al  (16) 
returned  to  Horsley  &  Clarke’s  electrophysiology  idea  by  using  anatomic  landmarks  and  an 
atlas  of  function  combined  with  patient-specific  information  from  images  to  allow  the 
placement  of  electrodes.  Two  reviews  of  stereotaxy  can  be  found  elsewhere  (17,  18). 
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Figure  2:  CT-Enabled  Stereotactic  Frame  and  CT  slice  showing  N-Bars 


With  general  charts  proving  to  be  unacceptable,  stereotaxy  had  to  await  an  improved 
method  for  determining  locations  within  the  brain.  The  development  of 
pneumoencephalograms  and,  later,  iodine-based  contrast  agents  for  ventriculography  started 
the  process  of  patient-specific  localization.  Now  structures  within  an  individual's  brain 
could  be  visualized,  and  landmarks  and  distances  determined  for  each  subject.  A  surgeon 
could  visualize  specific  anatomy  in  his  patient.  The  use  of  radio-opaque  materials  for 
constructing  the  frame  allowed  it  as  well  structures  within  the  brain  to  be  seen  in  the 
images.  But  the  pneumoencephalography /ventriculography  technique  was  not  without  its 
drawbacks.  Because  both  types  of  images  are  shadowgrams,  or  projection  images,  they 
compress  information  about  a  three-dimensional  object  onto  a  two-dimensional  plane.  If  a 
point  could  be  uniquely  determined  in  both  anterior-posterior  and  lateral  films,  then  a  probe 
could  be  guided  to  that  point  by  a  series  of  frame  adjustments.  The  ability  to  exacdy  define 
the  point  of  surgical  interest  in  both  films  was  in  no  way  guaranteed.  In  the  absence  of 
exacdy  definable  points,  the  surgeon's  experience  and  knowledge  of  general  anatomy  could 
help  approximate  the  site  of  surgical  interest,  the  general  case  combined  with  specific 
information  helping  to  better  approximate  the  individual  case.  If  the  target  were  visually  or 
electrically  distinguishable  from  the  rest  of  the  anatomy,  then  the  site's  location  could  be 
refined  from  the  rough  position  determined  by  the  frame.  The  conceptual  breakthrough  was 
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that  the  ventriculogram  provided  surgeons  with  patient-specific  information  to  modify  their 
three-dimensional  understanding  of  general  neuroanatomy. 

Once  localization  and  targeting  techniques  had  been  developed,  stereotactic 
methodologies  were  invented  for  a  variety  of  therapeutic  procedures.  As  was  the  case  with 
the  electrode  placement,  therapeutic  protocols  followed  protocols  designed  to  gather 
information.  Once  it  had  been  shown  that  a  narrow-guage  needle  could  be  placed  at  the 
tumor  site  with  relative  accuracy,  local  radiation  therapy  by  the  stereotactic  placement  of 
radiation  sources  became  possible  (19,  20).  Currently,  there  is  an  array  of  different 
radioactive  materials  that  may  be  placed  at  the  target  site  (21—24).  To  some  degree,  different 
materials  are  selected  for  different  tumor  types,  so  the  biopsy  procedure  often  precedes 
placement.  Recently  there  has  been  an  expansion  of  interventional  efforts  to  place 
therapeutic  objects  direcdy  in  the  lesion.  These  include  controlled  release  chemotherapy 
agents  (25). 

To  accurately  assess  the  efficacy  of  chemotherapy  and  radiation  therapy  protocols  for 
deep-seated  tumors,  histologic  confirmation  of  the  neoplasm  type  is  needed.  Furthermore, 
biopsy  may  be  pivotal  in  the  selection  of  the  most  appropriate  course  of  treatment.  Surgical 
biopsy  by  craniotomy  subjected  patients  to  most  of  the  risks  of  surgical  resection. 
Stereotactic  biopsy  was  developed  as  a  gready  less-invasive  procedure  for  capturing  tissue  for 
histological  examination  (26—30).  Once  the  neoplasm  type  is  exacdy  defined,  then  the  most 
appropriate  form  of  therapy  can  be  prescribed. 

Along  with  resection,  in  which  tissue  is  removed  from  the  brain,  stereotaxy  can  be  used 
in  transplanting  tissue  into  the  brain.  The  transplantation  of  adrenal  medullary  tissue  into 
the  brain  for  the  alleviation  of  Parkinson’s  disease  symptoms  is  a  surgical  procedure  of  both 
great  promise  and  great  debate  (31—33).  One  factor  that  has  led  to  the  uneven  success 
among  various  sites  is  the  variability  of  the  implant  location.  Stereotactic  techniques  can  be 
used  to  control  that  variability,  helping  to  place  implants  of  glandular  tissue  in  specific, 
consistent  locations,  thus  reducing  the  risk  of  the  procedure.  Stereotactic  techniques  were 
also  developed  for  external  beam  radiation.  In  1951,  Leksell  (34)  introduced  the  concept  of 
radiosurgery,  i.e.  the  use  of  targeted  external  beam  radiation,  in  which  the  dose  delivered  to 
the  target  is  built  by  the  summation  of  beams  from  a  variety  of  angles.  This  method  allows  a 
greater  dosage  to  be  delivered  to  the  target  while  reducing  that  to  the  rest  of  the  tissue. 
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Modem  tomographic  methods  have  allowed  better  definition  of  tumor  margins,  and  new 
techniques  for  intensity-modulated  radiation  therapy  allow  for  the  “sculpting”  of  the 
delivered  dose  (35,  36). 

ADVENT  OF  IMAGE-GUIDED  PROCEDURES 

The  creation  of  computed  tomography  (CT)  provided  the  surgeon  with  a  three- 
dimensional,  patient-specific  atlas.  Although  a  number  of  researchers  used  this  new 
technique  with  frames  (37—40),  and  techniques  were  developed  to  merge  CT  and  plane  film 
data  (41),  the  availability  of  three-dimensional  data  coupled  with  a  rapidly  advancing 
computer  field  made  new  approaches  possible.  In  the  mid  to  late  1980s,  a  number  of 
researchers  began  to  develop  systems  that,  although  varying  greatly  in  implementation, 
shared  a  common  revolutionary  idea:  track  the  surgical  position  in  the  physical  space  and 
display  position  in  image  space.  This  constitutes  a  reversal  of  classic  stereotaxy,  in  which  the 
physical  location  is  determined  from  the  images.  Dubbed  frameless  stereotaxy  by  some  (42- 
-44)  and  interactive,  image-guided  neurosurgery  by  others  (45,  46),  the  process  had  three 
major  components: 

1.  A  three-dimensional  spatial  localizer.  This  device  could  be  freely  moved  in  the 
operating  room,  and  the  location  and  trajectory  of  its  tip  dynamically  tracked.  Thus,  the 
device  would  return  a  position  triplet  <xp,yp,zp>  for  the  space  defined  by  its  motion. 

2.  A  registration  technique.  As  with  previous  stereotaxy,  the  relationship  between 
the  space  defined  by  an  extracranial  device  and  locations  seen  in  the  image  space 
<xi,yi,zi>  must  be  determined.  Instead  of  mapping  image  location  into  frame 
adjustments,  <xi,yi,zi>  =>  <xp,yp,zp>,  the  localizer  position  is  mapped  into  image 
space  <xp,yp,zp>  =>  <xi,yi,zi>  and  that  point  is  displayed  on  the  appropriate  image  or 
images.  This  means  that  all  image  information  is  retained  and  dynamically  displayed  as 
the  surgeon  moves  the  localizer.  Thus,  surgeons  are  able  to  determine  not  only  their 
present  surgical  position  but  the  position  of  all  perceptible  anatomic  structures. 

3.  A  means  of  displaying  the  location  in  image  space.  The  timing  of  the 

development  of  systems  that  tracked  surgical  location  in  image  space  was  not 
accidental.  Computers  capable  of  storage,  recall,  and  display  of  large  medical  image  sets, 
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and  yet  of  a  size  and  cost  appropriate  for  operating  rooms,  were  not  available  until  the 
1980s. 

The  first  system  in  press  was  the  tracked  microscope  system  of  Roberts  et  al  (47,  48) 
from  Dartmouth,  followed  quickly  by  a  system  developed  in  Tokyo  (49,  50).  It  is 
appropriate  that  the  first  two  systems  used  two  distinctly  different  methods  for  surgical 
location.  The  Dartmouth  system  used  a  sonic  triangulation  method  for  localization  whereas 
the  Tokyo  system  used  an  articulated  arm.  Those  two  approaches  represent  the  dominant 
three-dimensional  localization  paradigms,  as  explained  below. 

FUNDAMENTALS  OF  IMAGE-GUIDED  PROCEDURES 

IGPs  have  four  basic  components:  image  acquisition,  image-to-physical-space 
registration,  three-dimensional  tracking,  and  display  of  imaging  data  and  location.  In 
addition,  some  registrations  and  displays  require  segmentation  techniques. 

Image  Acquisition 

IGPs  require  dealing  with  five  different  types  of  images: 

1.  Projective  images  such  plane  films  and  angiograms;  tomographic  sets  such  as  computed 
tomography  (CT),  magnetic  resonance  imaging,  (MRI),  and  emission  tomograms  (SPECT, 
PET). 

2.  Intraoperative  two-dimensional  imaging;  ultrasound  image,  which  creates  slices  but 
which  a  number  of  researchers  are  combining  into  volumes. 

3.  Laparoscopic,  endoscopic,  and  microscopic  images,  which  are  projective  and  may  have 
distortions; .  intraoperative  plane  film  imaging,  which  both  projective  and  has  distortions 
associated  with  the  image  intensifier. 

4.  Intraoperative  tomograms,  including  both  CT  and  “open-frame”  or  “therapeutic”  MRI. 

5.  Processed  images,  including  rendered  images  that  may  represent  an  extracted  anatomic 
surface,  and  images  such  as  maximum  intensity  projections  from  tomographic 
angiograms. 

Although  the  history  of  stereotaxy  is  dominated  by  projection  images,  in  IGP  they  are 
rarely  used.  This  is  due  to  two  issues.  First,  in  order  to  form  a  registration,  six  reference 
points  (fiducials)  are  required  to  be  located  in  image  space  and  physical  space.  Tomograms 
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require  only  three  to  be  found  (51,  52).  Second,  a  projective  image  has  significant  spatial 
uncertainty  in  the  third  dimension. 

At  Montreal  Neurologic  Institute,  there  is  a  history  of  the  acquisition  and  use  of  stereo 
pair  angiograms  (53).  This  history  carried  over  into  the  stereotactic  and  IGP  system 
developed  there  (54,  55).  By  obtaining  stereo  images,  the  depth  uncertainty  can  be  reduced. 
That  simultaneously  obtained  projection  images  can  reduce  spatial  uncertainty  has  also 
been  shown  with  “C-arm”  intraoperative  imaging  and  fluoroscopy  (56);  The  pincushion  and 
S  distortion  in  the  image  intensifiers  can  be  corrected  (57),  and  if  homologous  points  can  be 
unambiguously  identified  in  both  images  then  the  point  can  be  identified  in  three 
dimensions. 

Tomographic  scans  have  been  used  since  the  initial  development  of  IGP.  Both  Roberts  et 

al  (47)  and  Watanabe  et 
al  (49)  displayed  surgical 
position  on  a  single 
tomogram  slice. 

Galloway  et  al  (58) 
demonstrated  a  four- 
window  display  (each 
512  x  512  pixels)  with 
image  sets  selectable 
from  various 
tomographic  modalities. 
These  images  could  be 
displayed  in  the  native 
orientation,  or 

reconstructed  sets  in  the  other  cardinal  planes  could  be  shown. 

Figure  3:  Cardinal  plane  view. 

Guthrie  (59)  and  Smith  et  al  (60)  developed  systems  in  which  the  original  image  volume 
was  resliced  to  show  the  plane  perpendicular  to  the  surgical  trajectory.  Perhaps  the  first  real 
demonstration  of  the  power  of  using  modem  tomographic  sets  quantitatively  was  in  a 
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surgery  performed  to  separate  conjoined  twins  (61).  Here  the  data  was  used  not  only  for 
qualitative  assessment  and  point  localization,  but  also  for  distance,  surface,  and  volume 
determination.  Although  these  results  were  not  directly  linked  to  the  procedural  guidance, 
the  success  of  the  procedure  demonstrated  the  true  value  of  image  analysis,  image 
processing,  and  data  extraction  in  guiding  a  therapeutic  process.  These  processes  allowed  the 
determination  of  the  volume  of  shared  tissue,  the  amount  of  skinn  needed  for  grafting  and 
allowed  surgeons  to  simulated  different  approaches  and  resection  strategies. 

There  are  two  major  difficulties  of  using  preoperative  tomograms  as  an  exclusive  form  of 
guidance.  The  first  is  that  the  tomogram  requires  that  the  surgeon  look  up  from  the 
surgical  view  and  blend  it  with  the  tomographic  orientation.  This  difficulty  has  been 
addressed  in  a  number  of  ways.  In  1993,  a  concept  called  enhanced  reality  was  pioneered, 
in  which  rendered  objects  from  a  tomographic  data  set  were  projected  into  the  surgical 
viewpoint  and  blended  into  the  visual  display  (62,  63).  This  concept  was  furthered  by 
injecting  that  blended  image  into  an  eyepiece  of  a  surgical  microscope  (64).  There  are 
significant  challenges  to  these  systems  in  terms  of  not  overwhelming  the  visual  system  of  the 
viewer. 

Mainwaring  (65)  pioneered  a  different  approach,  in  which  a  tracked  endoscope  serving  as 
a  surgical  pointer  had  its  position  displayed  on  a  computer  monitor.  By  placing  the 
computer  monitor  showing  the  tomographic  position  next  to  the  video  monitor  showing  the 
endoscope  images,  the  data  from  each  could  be  merged.  Stefansic  et  al  (66,  67)  have  shown 
derivative  systems  in  which  the  video  images  replace  one  of  the  windows  in  a  four-window 
display.  A  related  but  distinct  approach  uses  an  image  set  derived  from  a  tomogram  set.  In 
virtual  endoscopy  (68,  69),  a  tomographic  set  of  anatomical  structures  able  to  be  traversed  by 
endoscopes  is  segmented  and  rendered  in  such  a  way  that  a  path  mimicking  that  of  an 
endoscope  can  be  followed.  This  has  the  advantage  that,  unlike  with  a  true  endoscope,  it  is 
possible  to  see  “beneath”  the  surface  of  the  tissue  of  the  traversed  structure.  Such  a  virtual 
endoscope  is  also  able  to  get  past  blockages  that  may  foil  a  true  endoscope.  However,  such 
displays  cannot  provide  the  visual  clues  as  to  whether  tissue  is  healthy  or  not.  Therefore,  the 
combination  of  virtual-endoscopy  and  true-endoscopy  images  retains  the  power  of  both 
methodologies  (70,  71). 
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Three-Dimensional  Localization  Systems 

Localization  technologies  can  be  divided  into  three  classes:  geometric,  triangulation,  and 
inertial.  Although  there  have  been  some  attempts  at  using  inertial  systems  for  surgical 
localization  devices,  currently  such  system  are  unwieldy.  The  primary  confounding  property 
of  the  localization  task  is  that  it  is  actually  a  six-dimensional  task.  To  be  of  real  use,  not  only 
the  X,  Y,  and  Z  axes  but  also  the  spatial  orientation  of  a  device  must  be  tracked.  These 
orientations  can  be  represented  as  yaw,  pitch,  and  roll  Certain  geometric  techniques,  such 
as  placing  the  device  tip  along  a  device  axis,  can  reduce  the  problem  to  five  dimensions,  but 
for  inertial  systems  that  still  means  five  gyroscopes  or  gyroscope-equivalents. 

Thus,  the  field  of  localizers  can  be  reduced  to  two  classes  of  devices:  geometric  and 
triangulation.  As  mentioned  above,  these  two  classes  were  represented  by  the  first  two  IGP 
systems,  the  Dartmouth  and  Tokyo  systems.  Geometric  localizers  use  angle,  extension, 
and/or  bend  systems  to  sense  the  position  of  the  procedural  device.  Articulated  arms  using 
potentiomenters  or  optical  angle  sensors  are  the  most  common.  Beyond  the  Tokyo  system, 
researchers  from  the  University  of  Aachen  (72),  Vanderbilt  University  (73),  the  University  of 
Oolu  (74),  the  University  of  Alabama-Birmingham  (75),  and  Henry  Ford  Hospital  (76)  all 
developed  systems  using  articulated  arms.  In  addition,  one  of  the  first  commercially  available 
systems  from  ISG  used  an  articulated  arm  made  by  Faro  (Faro  Technologies,  Lake  May, 
FLA)  (77). 


Figure  4  -  The  two  Vanderbilt  Articulated  Arms 
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Articulated  arms  suffer  from  one  major  problem  as  surgical  localizers:  In  order  to  be 
accurate,  they  must  have  stiff  arms  between  the  rotational  joints.  That  requirement  limits 
how  light  the  device  can  be.  And  although  counter-weighting  can  make  the  device’s  end 
effector  seem  to  float,  such  techniques  add  to  the  mass  and,  thus,  to  the  inertia  of  the  device. 
It  is  because  of  this  constraint  that  most  developers  have  moved  away  from  articulated  arms 
as  surgical  localizers.  They  may  yet  have  applications  in  nonsurgical  IGPs,  such  as 
positioning  patients  for  radiotherapy. 

Although  the  use  of  articulated  arms  is  on  the  wane  for  image-guided  surgeries,  there  is  a 
growing  application  for  another  geometric  localization  technology.  In  a  number  of 
therapeutic  procedures,  for  example  otolaryngology  and  colon  polyp  biopsy  and  treatment,  it 
is  difficult  or  inappropriate  to  use  rigid  instruments.  Technologies  such  as  the  Shape  Tape™ 
allow  the  tracking  of  curvature  and  thus  the  end  point  of  the  devices  (78).  Although  the 
accuracy  of  such  systems  is  not  what  it  needs  to  be  for  IGP,  active  development  is  under 
way. 

The  second  major  type  of  localization  systems  used  in  IGP  is  triangulation.  In  these 
systems,  an  emitter  or  emitters  broadcast  energy  to  a  series  of  detectors  at  known  locations. 
By  measuring  the  distance  or  angular  position  of  the  emitters,  the  emitter’s  location  can  be 
determined.  If  three  or  more  emitters  are  mounted  on  a  rigid  structure,  then  the  location  and 
orientation  of  that  structure  can  be  determined. 

In  the  pioneering  work  done  at  Dartmouth  (48)  and  then  in  later  work  done  at  the 
Cleveland  Clinic  (79),  the  triangulation  systems  were  sonic.  They  consisted  of  emitters 
mounted  to  the  microscope  (Dartmouth)  or  to  a  bipolar  cautery  device  (Cleveland  Clinic).  In 
both  systems,  multiple  spark-gaps  were  used  as  a  sound  point  source  and  the  time-of-flight 
(TOF)  between  source  and  detector  was  measures.  From  this  the  distance  was  determined 
and  the  location  triangulated.  The  acoustic  systems  had  two  problems,  both  related  to  the 
speed  of  sound.  Variations  in  the  speed  of  sound  led  to  errors  in  the  determination  of  the 
emitter  location  and,  thus,  the  object’s  location.  This  could  be  handled  by  a  method  called 
pilot  sparking  (80),  in  which  a  fixed  spark  gap  was  fired  periodically  and  variations  in  transit 
time  were  used  to  correct  for  changes  in  the  acoustic  propagation  velocity.  The  second,  and 
more  intractable,  problem  was  that  of  TOF.  In  the  time  from  the  first  event  to  the  last,  the 
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device  cannot  move  or  a  serious  localization  error  will  occur.  Therefore,  the  device  must 
remain  stationary  for  N  (TOF  +  TD),  where  N  is  the  number  of  emitters  and  TD  is  the 
timing  delay  between  spark  events  necessary  to  deal  with  reverberations.  If  you  presume  an 
average  emitter-detector  distance  of  2  m,  then  the  TOF  from  each  emitter  to  the  detector 
(in  STP  air)  is  6  ms.  This  requires  a  stationary  tracked  object  and  limits  update  rates  and 
numbers  of  emitters,  thereby  making  the  device  sensitive  to  shadowing  (81).  Shadowing 
occurs  when  the  handle  of  the  device  lies  between  the  emitters  and  the  detector,  thus 
blocking  localization. 

The  TOF  issue  vanishes  if  the  ultrasound  energy  is  replaced  by  electromagnetic  energy. 
With  electromagnetic  sources,  TOF  and  TD  effectively  become  zero  replaced  only  by  source 
and  detector  time  constants.  This  means  that  in  directional  sources,  such  as  optical  sources, 
the  number  of  emitters  can  be  high,  allowing  for  full  range  of  motion  without  object 
shadowing.  If  the  electromagnetic  emitters  are  radiofrequency  (82)  or  magnetic  (83,  84), 
which  are  effectively  omnidirectional,  then  the  only  limitation  on  the  number  of  emitters  is 
their  size. 

Optical  devices,  both  active  (84-86)  and  passive  (87-89),  currently  dominate  the  IGP 
field.  This  is  perhaps  surprising,  given  the  need  for  a  continuous  line  of  sight  and  the 
directional  restrictions  of  the  optical  emitters  that  either  force  a  large  number  of  sources  or 
limit  the  orientation  of  the  tracked  device.  However,  optical  methods  may  be  winning  by 
default.  The  surgical  suite  contains  large,  conductive  structures,  such  as  the  operating  table. 
Because  of  the  motion  of  tools  and  conductive  fluids,  and  the  presence  of  electrically  active 
devices,  such  as  abalators,  imaging  devices,  and  patient-monitoring  equipment,  it  is  a 
dynamic,  conductive  environment.  All  these  factors  have  been  shown  (90)  to  confound 
radiofrequency  and  magnetic  localizers.  Active  work  continues  on  magnetic  localizers  (91), 
especially  for  tracking  devices  in  closed  patient  situations. 

REGISTRATION 

Three-dimensional  localization  and  tracking  devices  determine  position  in  physical  space. 
In  order  to  use  that  information  quantitatively  with  image  data,  a  registration  from  image 
space  to  physical  space  must  be  performed.  There  are  three  classes  of  registration 
techniques:  point  based,  feature  based,  and  volume  based.  Of  the  point-based  registration, 
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there  are  two  subtypes.  The  first  uses  anatomic  landmarks  or  features  as  intrinsic  reference 
points,  or  fiducials.  The  second  requires  the  attachment  of  extrinsic  objects,  or  fiducial 
markers.  The  differences  between  marker-based  and  feature-based  registration  are  relatively 
clear.  Because  the  markers  must  be  in  place  prior  to  the  acquisition  of  images,  the  decision 
to  use  marker-based  registration  must  be  made  before  acquisition.  For  this  reason,  marker- 
based  registration  is  said  to  be  a  prospective  method.  In  contrast,  feature-based  registration 
can  be  retrospective,  meaning  that  the  decision  to  perform  registration  can  be  made  after  the 
image  sets  are  acquired.  However,  feature-based  registration,  in  general,  requires  more 
computation  than  do  marker-based  methods,  and  they  are  prone  to  larger  error  (92).  In 
addition,  it  is  difficult  to  determine  the  quality  of  a  registration  arising  from  a  feature-based 
registration.  In  contrast,  there  are  statistical  methods  for  estimating  the  quality  of  a  marker- 
based  system. 

Point-Based  Systems 

In  point-based  systems,  points  are  localized  in  both  image  space  and  physical  space.  The 
locations  of  these  landmarks  in  the  two  spaces  can  be  arranged  as  lists  of  three-dimensional 
coordinates  and  a  three-dimensional  transformation  between  the  two  sets  determined  by  any 
one  of  a  number  of  mathematical  methods  (93).  In  the  early  image-guided  surgery  systems, 
anatomic  landmarks,  such  as  the  nasion,  the  tragi,  and  orbital  rims,  were  used  as  intrinsic 
fiducials  (49,  77,  94).  There  are  significant  difficulties  with  this  approach.  It  is  difficult  to 
define  an  anatomic  point,  and  even  could  such  a  point  be  located  in  physical  space,  the 
averaging  effect  of  finite-slice  thickness  introduces  a  spatial  uncertainty.  Evans  et  al  (95) 
demonstrated  that  by  overdetermining  the  problem,  e.g.  by  localizing  more  points  than  the 
minimum  number  necessary  to  determine  the  transformation  (three  for  three-dimensional 
images),  they  were  able  to  produce  transformations  that  were  more  robust  to  errors  in  the 
localization  of  any  given  point.  However,  if  the  localization  errors  among  markers  or 
features  are  uncorrelated,  then  overdetermination,  as,  for  example,  in  Evans’s  approach,  can 
be  employed  to  attenuate  the  effect  of  those  errors  on  the  calculated  transformation. 

The  benefit  of  increasing  the  number  of  homologous  points  leads  to  a  challenge  in  the  use 
of  anatomical  landmarks.  The  reason  that  images  of  a  patient  are  acquired  in  multiple 
modalities  is  that  they  provide  distinct  information.  Therefore,  almost  by  definition,  it  will  be 
difficult  to  accurately  locate  a  large  number  of  homologous  anatomic  landmarks  in  multiple 
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sets.  In  fact,  as  the  number  of  modalities  increases,  the  ability  to  find  the  same  anatomy  in 
each  decreases  dramatically.  Furthermore,  in  a  given  modality,  the  N  +  1  landmark  can 
always  be  expected  to  be  more  difficult  to  localize  than  the  N  landmark. 

In  physical-space  localization,  these  same  problems  arise.  As  with  image  localization,  if 
the  localization  errors  are  uncorrelated,  they  can  be  minimized  by  overdetermination. 
However,  in  attempting  to  localize  significant  numbers  of  points,  it  quickly  becomes  difficult 
to  continue  to  determine  structures  that  can  be  adequately  localized  in  both  image  space  and 
physical  space.  Significant  work  has  gone  into  using  landmarks  on  the  upper  row  of  teeth 
(96),  but  they  are  poorly  visualized  on  MRI  and  not  at  all  in  positron  emisssions 
tomography. 

The  need  for  large  numbers  of  landmarks  arises  because  of  the  difficulty  of  accurately 
localizing  a  single  landmark.  An  alternative  approach  is  to  construct  a  landmark,  or 
“marker,5'  that  is  designed  specifically  for  accurate  localization  and  then  to  attach  it  to  the 
patient.  Such  markers  are  termed  fiducial  markers.  In  using  anatomic  landmarks  as  fiducials, 
researchers  attempted  to  use  them  as  intrinsic  reference  points.  The  difficulty  is  that  there 
are  very  few  anatomic  points,  and  even  in  those  structures  that  might  come  to  a  point,  the 
partial  volume  effect  of  tomographic  imaging  blurs  their  location  into  a  voxel-sized 
resolution  cell.  The  difficulty  is  that  the  user  has  only  general  apriori  knowledge  about  the 
object  he  or  she  is  attempting  to  localize.  This  handicap  severely  limits  the  mathematical 
tools  that  can  be  brought  to  bear  to  improve  spatial  localization  to  less  than  a  voxel  size. 
However,  if  there  is  knowledge  about  a  structure  and  understanding  of  how  the  imaging 
process  works,  then  the  object  can  be  localized  in  space  to  a  much  higher  accuracy. 


Most  researchers  developing  systems  for  IGPs  have  used  extrinsic  fiducials  at  some  time. 
Common  early  practice  included  surgical  staples  or  small  ball  bearings  for  CT  (97,  98)  and 
capsules  filled  with  oil  or  fat  for  MRI  (99).  However,  the  staples  caused  slight  star  artifact  in 
CT,  making  it  difficult  to  localize  a  part  of  the  staple  for  use  in  identification  in  physical 
space,  and  the  center  of  the  ball  bearing,  although  easy  to  localize  in  image  space,  was 
difficult  to  localize  in  physical  space.  The  oil-filled  and  fat-filled  capsules  exhibited  chemical 
shift  of  up  to  1  cm  in  the  MRI  unit,  making  them  inappropriate  for  localization  tasks  (100). 
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As  the  concept  of  extrinsic  fiducials  grew,  the  physical  objects  (fiducial  markers)  that 
embodied  the  fiducials  became  more  sophisticated.  The  markers  could  be  made  to  be  visible 
in  multiple  image  modalities,  and  the  mechanisms  by  which  physical  space  localization  was 
accomplished  increased  in  accuracy.  From  simple,  in-house-created  objects  to  commercially 
developed  and  marketed  marker  systems,  a  number  of  fiducial  markers  are  available.  In 
general,  they  break  down  into  one  of  two  classes:  skin-surface  attachment  or  bone- 
implanted  markers.  The  division  between  the  classes  is  clear:  Skin-surface  markers  are  much 
less  invasive  than  bone  implant  markers,  but  the  skin-surface  markers  contain  a  potential 
source  of  error  due  to  skin  motion.  This  motion  may  be  normal  to  the  skull  surface,  due  to 
skin  swelling  or  skin  drying,  or  it  may  be  tangential,  due  to  tractions,  such  as  placing  the  head 
in  a  Mayfield  head  clamp  prior  to  surgery  (101). 


Figure  5  —  Acustar  markers  and  CT  showing  Marker 


There  are  two  commercial  realizations  of  bone-implanted  markers:  the  Acustar  system, 
developed  by  Vanderbilt  University  and  Johnson  &  Johnson  Professional  Inc  (102),  and  one 
developed  by  Leibinger  (103),  now  a  part  of  Stryker  Inc.  Implantation  of  the  bone-implant 
markers  is  analogous  to  minor  dental  surgery.  The  skin  surface  is  cleaned,  and  a  local 
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anesthetic  is  applied  to  the  scalp.  A  small  incision  is  made,  and  the  marker  posts  are  screwed 
into  the  outer  table  of  the  skull.  In  the  case  of  the  Acustar  markers,  a  flat-bottom  hole  is 
drilled  and  the  mounting  post  screwed  into  the  hole.  In  the  case  of  markers  from  Leibinger, 
the  markers  mount  on  a  self-tapping  titanium  post  that  is  screwed  into  the  outer  tablature  of 
the  skull.  The  Acustar  imageable  markers  are  cylinders,  7  mm  in  diameter  and  5  mm  high, 
filled  with  a  water  solution  containing  iodine  and  gadolinium,  designed  to  be  visible  in  CT 
and  a  wide  range  of  MRI  sequences.  The  size  guarantees  that  at  any  orientation,  the  marker 
is  visible  in  more  than  one  tomogram  slice  of  thickness  less  than  5  mm.  To  date,  these 
markers  have  been  tested  with  T1-,  T2-,  and  proton-density-weighted  spin-echo  sequences, 
as  well  as  magnetic  resonance  angiography  and  gradient-echo  images,  and  they  are  accurately 
localizable  in  all  of  them  (104).  The  Leibinger  marker  system  consists  of  a  titanium  screw,  a 
base  component,  and  an  imaging  marker.  The  imaging  marker  has  a  2-mm-diameter 
spherical  volume,  which  can  be  filled  with  a  contrast  agent.  A  more  complete  review  of 
markers  is  available  in  a  paper  by  Fitzpatrick  &  Galloway  (105). 

Feature-Based  Systems 

The  retrospective  nature  of  the  use  of  intrinsic  features  continues  to  appeal.  Although 
difficulties  in  identifying  anatomic  points  remain,  by  localizing  surfaces  or  features,  the  need 
for  one-to-one  correspondence  is  removed,  and  the  use  of  a  large  number  of  points  creates 
an  averaging  effect  that  reduces  uncorrelated  localization  noise.  A  number  of  researchers 
have  tried  to  use  skin-surface  information,  both  by  contact  (106)  and  by  optical  means 
(107,  108).  The  problem  with  these  techniques  is  that  the  skin  surface  is  a  dynamic 
structure,  responding  to  both  preoperative  drug  regimens  and  intraprocedural  tractions.  For 
the  head,  this  problem  has  been  addressed  by  using  the  skull  surface  as  a  registration  feature, 
capturing  intraprocedural  data  by  use  of  a  tracked  A-mode  transducer  to  localize  the  skull 
surface  transcutaneously  (109).  Others  have  used  features  on  the  brain  surface  for 
intraprocedural  registration  (110,  111).  However,  the  brain  changes  shape  once  the  skull  is 
opened,  with  the  greatest  changes  being  at  that  opening  (112). 

For  applications  other  than  intracranial,  surface  information  has  been  used  more 
successfully.  Two  groups  (113-115)  have  examined  surface  registration  in  the  spine,  either 
by  touching  points  on  the  vertebral  surface  or  by  tracked,  transcutaneous  ultrasound.  In 
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addition,  two  groups  have  used  the  organ  surface  in  liver  surgery  (116,  117).  A  point-based 
registration  is  usually  used  to  provide  an  initial  estimate  for  the  surface  registration. 

The  most  common  algorithm  for  matching  a  cloud  of  acquired  points  and  a  derived 
surface  is  the  ICP  (iterative  closest  point)  (118).  One  intriguing  aspect  to  this  method  is 
that  a  mix  of  surface  and  point  features  can  be  used.  Maurer  et  al  (1 1 9)  demonstrated  that  by 
using  such  a  mix,  a  better  registration  can  be  obtained  than  by  using  points  or  surfaces  alone. 

Volume-Based  Systems 

Although  volume-based  registration  has  proven  to  be  the  best  of  the  retrospective  image -to- 
image  methodologies  (120),  volumetric  registration  for  image-guided  therapy  has  proceeded 
slowly.  The  issue  here  is  that  if  a  high-quality  intraoperative  volumetric  data  set  can  be 
acquired,  why  use  preoperative  data  at  all?  In  certain  specialized  applications,  such  as  therapy 
for  prostate  cancer,  three-dimensional  intraprocedural  ultrasound  can  be  matched  to 
preoperative  tomographic  data  on  which  therapy  preplanning  has  occurred  (121). 

APPLICATION  EXAMPLES 

As  stated  at  the  beginning  of  this  article,  IGPs  grew  out  of  stereotaxy,  driven  by  both  the 
functional  importance  of  tissue  surrounding  the  site  of  surgical  interest  and  the  presence  of 
the  skull,  which  serves  as  a  static  mechanical  platform  for  the  soft-tissue  target:  the  brain. 
However,  IGPs  were  not  merely  improved  stereotaxy,  they  opened  the  way  for  new  modes 
of  thinking  about  therapy  delivery.  The  importance  of  specificity,  of  treating  the  lesion 
completely  with  as  little  damage  to  healthy  collateral  tissue  as  possible,  came  to  the  forefront. 
With  the  parallel  development  of  minimally  invasive  techniques  in  other  fields,  we  are 
beginning  to  see  a  convergence,  with  IGP  being  used  for  abdominal,  spinal,  and  other  form 
of  therapy. 

Intracranial  Resection/ Ablation 

Given  the  preponderance  of  intracranial  neurosurgery  and  otolargyngological  applications 
described  earlier,  it  would  take  an  article  much  bigger  than  this  to  cover  the  clinical 
applications  of  IGP  in  the  head  [for  more  complete  references  see  (122—125)].  However,  it 
is  here,  where  the  field  is  most  mature,  that  the  results  of  the  first  generation  of  work  and 
ideas  for  the  second  generation  can  be  seen.  For  cancer  cases,  the  ultimate  assessment  is,  do 
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the  patients  have  better  outcomes  with  IGP  than  with  previous  methods?  Perhaps  the  best 
data  for  that  comes  from  the  work  done  by  Berger  and  coworkers  (126,  127).  In  their 
papers,  significantly  better  outcomes  are  demonstrated  using  resections  to  imaging  defined 
margins  as  opposed  to  visually  defined  margins. 

In  addition,  the  presumption  that  tissue  is  rigid  has  been  discarded,  and  mechanisms  for 
correcting  for  periprocedural  tissue  deformation  are  coming  on  line.  These  include 
biomechanical  models  of  the  tissue  (112,  128, 129)  and  the  use  of  intraoperative  ultrasound 
imaging  to  gather  information  for  the  correction  of  preoperative,  high  signal-to-noise  images 
(130, 131). 

In  emerging  image-guidance  systems,  we  are  beginning  to  see  functional  mapping  play  a 
greater  role  than  in  first-generation  systems,  in  which  all  guidance  has  been  done  by 
anatomic  image  sets.  The  functional  information  is  coming  from  positron  emissions 
tomography,  functional  MRI,  and  “superbrain”  image  sets  in  which  a  functional  atlas  is 
defined.  In  the  case  of  the  atlas,  it  is  nonrigidly  registered  to  the  patient’s  image  set.  The 
resulting  deformations  can  then  be  applied  to  the  functional  data,  providing  a  probabilistic 
functional  map  of  the  patient.  It  is  in  the  placement  of  deep  brain  stimulators  that  this  work 
has  shown  the  most  growth. 

Beyond  purely  surgical  intervention,  IGPs  have  been  used  to  place  controlled-release 
chemotherapy  agents  in  brain  tumors  (132),  and  there  has  been  discussion  of  using  these 
techniques  for  placement  of  gene  therapy  agents  to  improve  their  efficiency  (133). 

Although  cancer  therapies  have  dominated  the  intracranial  applications,  IGPs  have  had  a 
role  in  vascular  surgeries  (134,  135),  and  image-guided  radiosurgeries  for  vascular 
abnormalities  have  become  commonplace  (136,  137). 

Functional  Neurosurgery  and  Stimulator  Placement 

The  use  of  electrophysiological  data  for  guidance  goes  back  to  the  beginning  of  stereotaxy. 
Interventional  applications,  such  as  ablations  of  foci  of  abnormal  electrical  activity  for 
interruption  of  epileptic  seizure  initiation  or  propagation  (138,  139),  and  ablations  of  deep 
gray  nuclei  for  alleviation  of  Parkinson's  disease  tremors  (140,  141),  used  imaging  to 
approximate  the  ablation  site  but  used  electrophysiological  recording  for  final  guidance. 
Additionally,  ablations  have  been  performed  for  the  relief  of  chronic,  previously  intractable 
pain,  such  as  that  arising  from  some  forms  of  cancer  (142, 143).  However,  such  techniques 
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were  limited  by  the  variability  of  anatomy,  distribution  of  function,  and  presence  of  space- 
occupying  lesions.  However,  two  recent  developments  are  spurring  additional  growth.  The 
first  is  the  use  of  implantable  stimulators  to  act  as  reversible  ablations  (144,  145).  This  means 
that  cellular  disruption  need  not  be  a  permanent  step.  The  second  advance  is  the  use  of 
image  volumes  as  atlases  to  allow  the  better  integration  of  data  across  patients  (146,  147).  In 
addition,  the  use  of  functional  MRI  as  a  data  source  for  IGPs  promises  to  provide  patient- 
specific  functional  information.  However,  continued  work  is  needed  to  fully  understand  the 
fMRI  images  and  to  validate  their  use.  Beyond  surgical  ablation  and  the  placement  of 
stimulators,  image-guided  radiosurgery  (1 48)  has  found  applications  in  functional 

Spinal  Applications 

Over  a  quarter  of  a  million  lumbar  spine  operations  will  be  done  in  the  United  States  this 
year.  The  federally  funded  Back-Pain  Patient  Outcomes  Assessment  Team  (HS  06344) 
placed  the  cost  of  back  pain,  the  leading  cause  of  loss  of  work  days  in  the  United  States,  at 
$50  billion.  Beyond  the  lumber  problems,  cervical  spine  fractures  and  cystic,  vascular,  and 
neoplastic  diseases  of  the  spinal  cord  add  a  significant  number  of  cases  each  year.  The  nature 
of  the  spine  represents  a  substantial  challenge  for  image -guided  techniques.  The  spine 
cannot  be  presumed  to  be  monolithically  rigid;  rather,  it  is  piece-wise  rigid,  with  deformable 
structures  between  the  rigid  segments.  This  requires  a  system  in  which  multiple  registrations 
can  be  performed.  Beyond  the  vertebral  and  disk  structures,  the  spine  is  a  web  of  ligaments 
and  tendons  with  intertwined  vascular  structures  and  functionally  vital  nerve  roots.  The 
challenge  is  so  complex  that  in  1999  a  workshop  was  held  to  try  to  clarify  the  technical 
requirements  for  image-guided  spine  procedures  (149). 

One  of  the  leading  sources  of  failure  in  surgery  to  alleviate  pain  or  structural  failure  is  the 
change  in  the  length  and  mechanics  of  the  spine  following  surgical  intervention.  A  recent 
advance  in  the  use  of  struts  called  spinal  instrumentation  is  replacing  direct  vertebral  fusion. 
Often,  to  rigidly  attach  the  instrumentation,  the  device  is  fixed  to  the  vertebrae  via  the  use 
of  screws  driven  into  the  pedicle  of  the  vertebrae.  Because  the  spinal  cord,  nerve  roots,  and 
supporting  vasculature  all  lie  within  millimeters  of  the  pedicle,  accurate  guidance  is  essential. 
Image-guidance  techniques  using  preoperative  tomograms  and  intraoperative  imaging  such 
as  dual  fluoroscopy  have  been  developed  (150, 151). 
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Although  tumors  of  the  spine  represent  a  lesser  health  care  issue  than  does  back  pain  or 
trauma,  the  occurrence  is  devastating  to  the  individual.  IGPS  both  surgical  (152,  153)  and 
radiosurgical  (1 54)  are  showing  promise  for  more  specific  resection. 

Orthopedics 

Although  neurosurgical  applications  were  driven  by  the  functional  importance  of 
intervening  tissue  between  the  skull  and  the  lesion,  orthopedic  applications  were  driven  by 
the  need  for  accurate  and  precise  control  of  position  and  angle.  It  is  the  need  for  precise 
positioning  of  position  and  angle  that  opens  the  doors  for  robotic  IGPs.  Although  robots 
have  been  used  in  surgery  for  a  while  (155,  156),  their  role  has  been  essentially  device 
holder.  The  Robodoc  system  (157,  158),  with  an  active  head,  is  essentially  an  image-guided, 
numerically  controlled  milling  machine.  This  allows  the  very  accurate  and  precise  placement 
of  artificial  hip  components  both  in  the  femur  and  on  the  pelvis.  Beyond  hip  replacement 
procedures,  additional  orthopedic  applications  include  use  in  acetabulature  fractures  (159) 
and  in  osteotomies  (160). 

Abdominal 

As  IGPs  prove  their  worth  in  intracranial,  spinal,  and  orthopedic  applications,  their 
potential  value  in  abdominal  surgeries  becomes  more  apparent.  Two  of  the  early  targeted 
areas  have  been  the  liver  and  the  prostate.  In  both,  the  major  challenge  is  registration. 

Unlike  the  skull,  the  vertebrae,  or  the  long  bones,  in  liver  and  prostate  there  can  be  no 
presumption  of  even  piece-wise  rigidity. 

In  liver  surgery,  intraoperative  ultrasound  has  become  the  standard  for  image  guidance 
(161,  162).  However,  ultrasound  is  a  low  signal-to-noise  imaging  modality,  and  it  is  difficult 
to  localize  the  center  of  a  three-dimensional  target.  Recent  work  (163,  164)  is  bringing  three- 
dimensional  localization  and  guidance  into  liver  surgery.  Additionally,  image-guided 
cryosurgery  (ablation  of  tissue  with  extreme  cold)  (165)  and  interstitial  photon  radiation 
therapy  (166)  have  been  applied  to  the  liver. 

Prostate  cancer  is  one  of  the  most  commonly  occurring  forms  of  cancer,  and  new  blood 
chemistry  studies  have  increased  the  detection  rate  of  that  cancer.  However,  optimum 
treatment  remains  difficult.  Transperineal  approaches  have  been  developed,  but  patient 
posture  and  bladder  fullness  can  cause  significant  changes  in  prostate  position  and 
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orientation.  Intraprocedural  imaging,  such  as  transrectal  ultrasound  (167—169)  and 
intraprocedural  MRI  (170),  have  helped  improve  guidance.  Interstitial  brachytherapy  (169, 
171)  has  been  shown  to  be  effective  and  relatively  safe.  However,  techniques  need  to  be 
developed  to  correct  for  changes  in  the  prostate  during  brachytherapy  seed  placement  (172) 
and  for  deflection  of  the  radioactive  seed  placement  needle. 

CONCLUSIONS 

Although  there  has  been  an  explosion  of  interest  in  image-guided  procedures  (IGPs)  over 
the  past  decade,  and  a  single  review  article  is  defined  as  much  by  what  it  leaves  out  as  what  it 
contains,  we  are  still  very  much  in  the  first  generation  of  IGPs.  For  the  most  part,  we  deal 
with  tissue  as  if  it  is  monolithic  and  rigid,  we  guide  procedures  based  on  anatomy  rather  than 
function,  and  we  are  crudely  integrating  image  information  over  time  and  space.  The  biggest 
question,  do  IGPs  produce  better  outcomes,  is  only  now  being  addressed.  However,  there 
are  some  signs  that  we  can  point  to.  Once  implemented  on  a  regular  basis,  IGPs  take  less 
operating  room  time  than  do  equivalent  stereotactic  procedures  (173),  IGPs  are  rapidly 
being  integrated  with  minimally  invasive  procedures  to  reduce  patient  discomfort  and 
recovery  time  (174),  and  IGPs  are  enabling  technologies,  allowing  the  possibility  of 
successful  treatment  where  it  was  not  possible  before. 

Having  demonstrated  that  image-guided  procedures  can  be  made  to  work  and  having 
some  evidence  that  they  have  possitive  clinical  outcomes,  considerable  effort  needs  to  be 
placed  into  validation  and  assessment  of  systems  and  techniques.  One  of  the  difficulties  is 
comparing  published  results  of  IGP’s  is  that  system  performance  is  not  measured  in  a 
consistant  manner.  For  example,  some  users  of  localizers  report  precision  where  some 
report  accuracy.  There  is,  as  of  yet,  no  satisfactory  way  to  assess  the  accuracy  of  a  surface- 
based  registration  using  only  the  two  sets  of  surface  points.  The  field  needs  independent 
assesment  measures  perhaps  developed  through  the  National  Institute  for  Standards  and 
technology.  If  these  tests  can  be  developed  and  agreed  upon,  then  when  a  new  technology 
or  process  emerges,  it  can  be  assessed  on  a  level  playing  field. 

The  maturation  of  IGPs  requires  a  change  in  research  teams  as  well.  Any  system  where 
the  design  engineer  presumes  to  understand  a  procedurist’s  needs  and  builds  a  device  to 
those  standards  is  doomed  to  failure.  Correspondingly,  the  field  has  demonstrated  that  a 
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successful  device  or  process  requires  a  level  of  technical  sophistication  beyond  that  of 
physicians  working  on  it  in  their  spare  time.  Successful  creation,  development,  and 
refinement  and  appropriate  testing  of  IGPs  is  going  to  require  teams  of  physicians, 
engineers,  computer  scientists,  nurses,  and  clinical  trial  and  outcome  experts.  However,  the 
potential  payoff  is  enormous.  By  guiding  therapy  to  where  it  is  needed,  and  by  avoiding 
healthy  structures,  both  the  process  and  the  outcome  are  improved. 
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Image-guided  Surgery  and  Therapy:  Current  Status  and  Future 

Directions 
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ABSTRACT 

Image-guided  surgery  and  therapy  is  assuming  an  increasingly  important  role,  particularly  considering  the 
current  emphasis  on  minimally-invasive  surgical  procedures.  Volumetric  CT  and  MR  images  have  been  used 
now  for  some  time  in  conjunction  with  stereotactic  frames,  to  guide  many  neurosurgical  procedures.  With 
the  development  of  systems  that  permit  surgical  instruments  to  be  tracked  in  space,  image-guided  surgery 
now  includes  the  use  of  “frame-less”  procedures,  and  the  application  of  the  technology  has  spread  beyond 
neurosurgery  to  include  orthopedic  applications  and  therapy  of  various  soft-tissue  organs  such  as  the  breast, 
prostate  and  heart.  Since  tracking  systems  allow  image-guided  surgery  to  be  undertaken  without  frames,  a 
great  deal  of  effort  has  been  spent  on  image-to-image  and  image-to-patient  registration  techniques,  and 
upon  the  means  of  combining  real-time  intra-operative  images  with  images  acquired  pre-operatively.  As 
image-guided  surgery  systems  have  become  increasingly  sophisticated,  the  greatest  challenges  to  their 
successful  adoption  in  the  operating  room  of  the  future  relate  to  the  interface  between  the  user  and  the 
system.  To  date,  litde  effort  has  been  expended  to  ensure  that  the  human  factors  issues  relating  to  the  use 
of  such  equipment  in  the  operating  room  have  been  adequately  addressed.  Such  systems  will  only  be 
employed  routinely  in  the  OR  when  they  are  designed  to  be  intuitive,  unobtrusive,  and  provide  simple 
access  to  the  source  of  the  images. 

Keywords:  Image-guided  surgery;  image-guided  therapy,  registration,  tracking,  robotics,  virtual  reality, 
simulation,  software 


1.  INTRODUCTION: 

Image-guided  surgery  and  therapy  has  its  roots  in  the  field  of  stereotactic  neurosurgery.  While  this  approach 
was  not  specifically  “image-guided”  in  the  beginning,  it  nevertheless  provided  the  means  to  introduce 
probes  into  the  brain  at  precisely  defined  locations.  At  the  advent  of  truly  three-dimensional  imaging,  first 
with  CT  and  later  with  MRI,  volumetric  images  were  used  with  stereotactic  frames  to  guide  many 
neurosurgical  procedures.  Upon  the  development  of  systems  that  permitted  surgical  instruments  to  be 
tracked  in  space,  image-guided  surgery  began  to  include  the  use  of  “frame-less”  procedures,  and  the 
application  of  the  technology  spread  beyond  neurosurgery  to  include  orthopedic  applications  and  therapy  of 
various  soft-tissue  organs  such  as  the  breast,  prostate  and  heart. 

This  paper  begins  with  an  overview  of  stereotactic  technology  and  follows  with  examples  of  applications  in 
common  use  or  in  development.  Current  problems  and  limitations  of  current  practice  are  identified,  and  the 
paper  concludes  with  a  glimpse  into  the  future  of  image-guided  surgery. 
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2.  STEREOTACTIC  SURGERY 


2.1  Frame-based 

Computerized  planning  systems  made  their  debut  in  the  early  1980’s  with  simple  programs  that  established 
coordinate  systems  in  the  brain  based  on  frame-based  fiducial  markers.  This  approach  rapidly  evolved  to 
allow  images  from  multiple  modalities  to  be  combined  so  that  surgical  planning  could  proceed  using 
information  from  a  combination  of  MRI,  CT  and  angiographic  images.  Such  multi-modality  imaging  was 
considered  important  for  certain  procedures,  such  as  the  insertion  of  probes  or  electrodes  into  the  brain. 
The  ability'  to  simultaneously  visualize  the  trajectory  with  respect  to  images  of  the  blood  vessels  (either  using 
orthogonal  projections  or  stereoscopic  image  pairs),  enabled  the  pathway  to  be  planned  with  the  confidence 
that  it  traversed  within  a  vascular-free  zone.  1'2’25,26  Much  of  stereotactic  surgery  was  concerned  with 
procedures  involving  the  introduction  of  probes,  cannulae  or  electrodes  into  the  brain.  However,  in  1986, 
Kelly3'5  proposed  an  innovative  approach,  based  on  stereotactic  principles  for  the  treatment  of  tumors. 
Instead  of  a  narrow  cannula,  he  approached  a  tumor  volume  via  a  cylindrical  retractor  integrated  with  a 
stereotactic  frame.  Such  tumors  could  be  stereotactically  biopsied  and  treated  by  implantation  of 
radionuclide  sources,  or  resected  using  computer-assisted  stereotactic  laser  microsurgical  techniques.  Kelly’s 
work  was  a  bridge  between  conventional  frame-based  stereotactic  neurosurgery,  and  the  frameless 


2.2  Frameless  stereotaxy 

In  spite  of  the  advantages 
of  the  stereotactic  frame, 
most  involve  a  surgical 
procedure  to  fasten  them 
to  the  patient,  either 
through  the  creation  of  3-4 
1-2  mm  deep  holes  in  the 
skull  with  a  twist-drill  to 
accept  blunt  pins,  or  sharp 


Figure  1 .  Typical  Stereotactic  frame,  a)  fastened  to  the  patient’s  head  prior  to  surgery; 
b)  with  fiducial  marker  plates  attached  prior  to  the  scanning  procedure. 


pins  applied  to  the  skull 
under  pressure.  The  frame 
(or  in  some  cases  just  the 


base-ring  which  can  be  separated  from  other  structures)  may  also  present  unnecessary  clutter  in  the 
operating  room.  Hence,  there  is  a  general  desire  to  be  rid  of  the  frame  if  possible.  However,  without  the 
frame  to  provide  the  fiducial  markers,  some  other  sort  of  reference  system  must  be  employed  to  register  the 
patient  to  the  image(s).  Commonly  used  registration  methods  are  discussed  below. 


2.3  Point  matching 

The  most  common  method  of  registering  an  image  to  the  patient  involves  the  identification  of  common 
landmarks  both  in  the  images  and  on  the  patient.  For  example,  a  tracked  pointer  may  identify  landmarks 
such  as  the  outer  canthi  of  the  eyes,  the  tragus  of  the  ears,  and  the  nasion,  while  these  same  structures  are 
identified  within  the  three-dimensional  images  of  the  patient.  Unfortunately  there  is  bound  to  be  some 
variation  in  the  identified  locations  of  the  landmark  points  on  the  patient,  as  well  as  some  difficulty  in 
identifying  exacdy  the  same  locations  within  the  patient’s  three-dimensional  image.  Generally,  a  least- 
squares  approach  is  used  to  obtain  an  approximation  to  the  correct  registration.  However,  unless  all  of  the 
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registration  points  are  appropriately  distributed  about  the  surgical  site,  small  inaccuracies  in  registration  in 
the  region  containing  the  homologous  points  can  be  magnified  for  more  remote  points.  ' 

2.4  Surface  matching 

Some  systems  employ  a  surface-matching  approach  to  complement  the  point  matching  method.  This 
technique  involves  using  the  probe  to  sample  points  on  the  surface  of  the  patient,  and  then  determining  the 
best  match  of  this  point-cloud  to  an  extracted  surface  from  the  3-D  patient  image.  This  combined 
approach  using  both  points  and  surfaces  is  described  by  Maurer8  and  is  incorporated  in  several  commercial 
image-guided  neurosurgical  systems.  Under  ideal  conditions,  (i.e.  in  phantom  tests  where  homologous 
structures  are  easily  identified  and  there  is  no  movement  of  the  markers  with  respect  to  the  object),  accuracy 
approaching  that  of  stereotactic  frames  can  be  achieved.9  However,  under  clinical  conditions,  where  natural 
features  on  the  patient’s  skin  are  identified,  the  accuracy  obtainable  from  his  type  of  approach  decreases  due 
to  the  subjective  identification  of  homologous  point-pairs  on  the  patient  and  in  the  images.  While  this  may 
be  adequate  for  many  neurosurgical  purposes,  it  is  not  appropriate  for  procedures  requiring  great  precision. 

2.5  Bone-mounted  markers 

The  accuracy  and  precision  of  point  matching  procedures  can  be  improved  by  using  surface  markers  glued 
to  the  patient’s  skin.  In  this  case  their  location  can  be  more  precisely  determined  using  the  pointer,  and  they 
can  be  automatically  identified  within  the  patient’s  three-dimensional  images.  While  this  improves  the 
precision  of  the  matching,  there  remains  the  problem  that  skin  mounted  markers  can  move  with  respect  to 
the  underlying  bony  anatomy,  and  therefore  add  additional  error.  Maurer  et  alw  have  demonstrated 
convincingly  that  the  only  way  to  achieve  patient-to-image  registration  with  the  same  accuracy  as  can  be 
obtained  with  a  stereotactic  frame  is  by  using  bone-mounted  fiducial  markers.  Even  though  the 
implantation  of  these  reference  markers  constitutes  a  procedure  that  is  approximately  as  invasive  as  the 
installation  of  a  stereotactic  frame,  it  can  nevertheless  be  performed  under  local  anesthetic  and  represents  a 
level  of  invasiveness  that  is  minor  compared  to  the  surgical  procedure  that  is  to  follow.  Maurer  et  alw  show 
that  the  accuracy  and  precision  obtainable  with  properly  designed  markers  is  in  the  order  of  0.5  to  1.5mm. 

2.6  Tracking  Systems 

Tracked  probes  provide  the  key  to 
image-guided  surgery. 

Mechanical,9’11  ultrasonic, 

magnetic,12  or  optical13  methods 
have  all  been  employed  to 
determine  the  probe’s  position,  but 
the  use  of  optical  trackers  (both 
active  and  passive)  has  now  become 
almost  universal.  Until  recently 
there  was  no  satisfactory  means  of 
tracking  an  object  (for  example  a 
catheter  tip  or  a  flexible  endoscope) 
within  the  body  unless  one 
employed  a  real-time  imaging 
system  like  MRI,14’16  ultrasound,17  or 
fluoroscopy.  Recently,  miniature 
magnetic  sensors111  have  become 

patient  image  is  a  concatenation  of  these  transformation  matrices.  - 


Optical  tracker 


Mt 


LED  elements 
on  probe 


Figure  2.  Typical  instrument  tracking  scenario.  The  symbols  Mxx  represent  the 
transformation  matrices  relating  various  coordinate  systems.  The 
transformation  matrix  describing  the  relationship  between  the  probe  and  the 
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available  that  can  be  inserted  in  a  catheter,  which  opens  the  door  to  tracking  flexible  instruments  within  the 
body.  Most  of  the  devices  in  routine  use  currently  report  accuracies  and  precisions  of  better  than  1mm, 
which  is  generally  accepted  as  a  target  performance  standard  for  tracking  devices.  Whatever  process  is 
employed  at  the  registration  and  instrument  tracking  phases  of  the  procedure,  a  number  of  coordinate 
transformation  calculations  must  be  made  to  relate  the  coordinates  of  “image-space”  to  those  of  “patient 
space”,  as  illustrated  by  Figure  2. 


2.7  Intra-operative  imaging. 

Although  tracking  systems  allow  image-guided  surgery  to  be  undertaken  without  frames,  and  permit 
dramatic  expansion  in  the  scope  of  procedure  to  which  it  could  be  applied,  the  move  to  encompass  open- 
craniotomy  procedures  brought  other  problems.  The  absence  of  the  frame  compromised  the  accuracy  with 
which  the  images  could  be  registered  to  the  patient,  and  the  fact  that  the  tissue  could  move  during  the 
procedure  meant  that  the  pre-operative  images  did  not  necessarily  represent  the  state  of  the  tissue  accurately 
during  surgery.  A  great  deal  of  effort  has  been  spent  on  image-to-image  and  image-to-patient  registration 
techniques,  and  means  of  combining  real-time  intra-operative  imaging  with  images  acquired  pre-operatively. 
Approaches  to  this  problem  mclude  the  use  of  “open”  MR  magnets,  which  allow  the  patient  to  be  imaged 
dynamically  during  the  procedure,18’19  and  intra-operative  imaging  using  standard  magnets  that  can  be 
employed  to  image  the  patient  at  several  time-points  during  the  surgical  procedure.20  These  approaches  are 
expensive  and  somewhat  unfriendly  with  respect  to  the  OR  environment.  In  response  to  this  problem,  the 
use  of  intra-operative  ultrasound  has  been  employed  either  alone,21'22  or  in  conjunction  with  pre-operative 
MRI.23’24 


3.  MULTI-MODALITY  IMAGING 


Although  CT  was  the  primary  imaging  modality  originally  employed  for  stereotactic  surgical  planning,  it 

soon  became  evident  that  it  could  be  complemented  by  other  imaging  modalities.  In  particular,  the 

combination  of  MRI  and  digital  subtraction  angiography  provides  both  anatomic  and  vascular  information; 

while  MRI  (or  CT)  and  PET  (or  Spect  /  fMRI)  provided  anatomical  and  functional  data  in  the  same  image 

volume.  For  this  reason,  multi-modality  image  display  and  analysis  systems  were  developed  to  allow  the 

planning  and  guidance  operation  to  take  place  using  multiple  image  data  sets  simultaneously.25  27  Many  of 

these  techniques  are  discussed  in  a 

recent  comprehensive  review  on 
■  *  •  28 
image  registration. 

Although  MRI  and  CT  are  three- 
dimensional  imaging  systems  and 
digital  subtraction  angiography  is  two- 
dimensional,  it  is  nevertheless  possible 
to  display  the  projection  of  a  point 
identified  in  the  3-D  modality  within 
the  2-D  angiogram.  Likewise  a  point 
within  the  planar  angiogram  can  be 
represented  as  a  line  within  the  MRI 
or  CT  volume.  By  employing 
orthogonal  or  stereoscopic  angiogram 
pairs,  three-dimensional  localization 


b  http://www.ascension-tech.com/products/minibird 


Figure  3.  Stereoscopic  (crossed-eye)  view  of  lateral  (top)  and  anterior- 
posterior  (bottom)  angiograms  along  with  registered  MR  image.  A  3-D 
cursor  (arrows)  driven  through  the  volume  defined  by  the  angiograms  is 
tracked  in  the  appropriate  slice  of  the  MR  volume. 
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can  be  achieved  within  the  angiogram-defined  space  (Figure  3).  This  multi-modality  approach  was 
particularly  important  when  planning  trajectories  to  deep  brain  structures,  or  when  implanting  EEG 
recording  electrodes  during  a  procedure  to  localize  the  foci  of  epileptic  seizures.29,30  In  both  cases,  it  is 
important  to  ensure  that  the  trajectory  follows  a  path  that  is  free  from  major  blood  vessels.  This  can  be 
easily  achieved  by  verifying  the  trajectory  that  is  planned  on  the  basis  of  the  MRI  or  CT  images  by 
examining  its  projection  within  the  angiogram  images. 


3.1  Physiology 

Physiological  information  is  often  useful  as  a  complement  to  the  anatomical  data  furnished  by  MRI,  CT  or 
angiography.  In  addition  to  the  information  derived  from  PET  or  fMRI  some  neurosurgical  procedures 
(such  as  the  therapy  of  Parkinson’s  disease)  rely  heavily  on  electrophysiological  information,  recorded  from 
deep-brain  electrodes.  Mapping  the  electrophysiological  responses  recorded  during  such  procedures  onto 
the  3-D  anatomical  images  of  the  patient  helps  the  surgeon  navigate  towards  the  desired  target.  Moreover,  a 
database  of  such  responses,  normalized  through  non-linear  image  registration  to  a  standard  data  space,  can 
be  integrated  with  the  patient’s  3-D  MRI  to  assist  the  surgeon  by  predicting  the  likely  target  for  creating  a 
therapeutic  lesion31  (Figure  4). 

3.2  Video 

Intra-surgical  video  images  can  add  a 
great  deal  of  information  to  an  image 
guided  surgical  procedure.  The 
integration  of  the  real  and  modeled 
worlds  in  this  manner  is  often  referred 
to  as  augmented  or  enhanced  reality7 
visualization.  Kikinis32  and  Davey33 
demonstrated  the  efficacy  of  integrating 
a  video  image  of  the  operative  site  with 
the  graphical  representation  of  a  tumor 
or  other  target  structure.  In  this 
manner,  the  surgeon  can  “see- 
through”  the  patient’s  skin  into  the 
operative  site,  to  facilitate  the  planning 
of  an  optimal  approach  to  the  surgical 
target.  More  recently,  other 
investigators  have  achieved  similar 
results  by  integrating  virtual  images 
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Figure  4  Atlas  of  Facial,  Finger,  and  Foot  paresthesias  evoked  by 
microstimulation  (300Hz  stimulus,  <  100pA,  0.2ms)  derived  from  26 
patients.  Left:  Location  of  stimuli  coordinates  within  deep  brain  region 
of  individual  patient.  Right:  magnified  region  showing  somatatopy 
(facial  responses  in  white,  finger  responses  -  black,  foot  responses  - 
grey) 


(derived  from  3-D  medical  imaging)  into  the  same  space  occupied  by  the  physical  object,  through  the  use 
of  half-silvered  mirrors  suspended  above  the  operative  sitec,  or  head-mounted  displays.34,35  Using 
stereoscopic  visualization  and  appropriate  head-tracking,  this  approach  can  place  a  realistic  3-D  virtual 
image  of  the  organ  in  the  same  physical  space  occupied  by  the  actual  organ. 


3.3  Microscopes 

Since  many  surgical  procedures  involve  the  use  of  an  operating  microscope,  the  integration  of  the 
microscope  images  with  those  obtained  from  preoperative  MRI  and  CT  scans  is  a  natural  application  of 
augmented  reality.  In  this  case  the  images  from  a  tracked  microscope  can  be  integrated  electronically  in  a 
digital  or  analog  fashion,  with  the  combined  images  being  displayed  on  a  video  or  computer  monitor. 


c  http://www.mrcas.ri.cmu.edu/projects/overlay  /system.html 
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Alternatively,  the  images  from  the  preoperative  scans  can  be  projected  into  the  visual  field  of  the 
microscope.36  This  technology  has  also  been  incorporated  into  head-mounted  displays  where  the  position  of 
the  surgeon  wearing  such  a  display  is  tracked  and  the  preoperative  image  corresponding  to  the  viewpoint  of 
the  surgeon  is  projected  on  his  field  of  view.37 

3.4  Endoscopes 

While  an  operating  microscope  can  “see”  into  an  operating  site  through  a  skull  opening  greater  than  about  2 
cm  in  diameter,  when  we  seek  to  visuali2e  the  target  directly  through  a  smaller  opening,  other  means  must 
be  employed.  The  endoscope  has  been  used  for  many  years  to  gain  visual  access  to  body  cavities,  and  has 

been  employed  in  neurosurgery  to  assist  in 
surgical  procedures  within  the  ventricles.  A 
technique  that  has  become  popular  recently 
is  “virtual-  endoscopy,”38,39  which  uses 
computer  graphics  to  simulate  the  view  of 
an  endoscope  placed  in  a  particular  body 
cavity,  based  on  the  representation  of  the 
cavity  derived  from  preoperative  MRI  or  CT 
images.  With  the  increasing  emphasis  on 
minimally  invasive  surgery,  there  has 
recently  been  an  active  interest  in  combining 
the  video  images  from  standard  endoscopy 
with  the  computer-generated  images  of 
virtual  endoscopy.  The  aim  here  is  to  place 
the  endoscope  image  observed  during  a 
minimally  invasive  surgical  procedures,  in  its 
proper  context  by  merging  it  with  the 
equivalent  surface  extracted  from  the 
preoperative  images.  Several  authors  have 
recently  reported  on  the  clinical  application 
of  combining  images  of  the  ventricle  wall 
obtained  from  a  tracked  endoscope,  with  the  equivalent  images  from  CT  or  MRI.40  42  The  simplest  mode  of 
operation  of  this  approach  is  to  simply  track  the  tip  of  the  endoscope,  and  display  the  tracked  point  on 
orthogonal  image  slices  derived  from  the  original  data.  The  most  convenient  means  of  achieving  this  goal  is 
to  display  the  three  slices  that  intersect  at  the  tracked  point.  However,  in  order  to  place  the  endoscopic 
image  in  its  proper  context,  a  surface  rendition  of  the  cavity  within  which  the  endoscope  resides  is 
constructed,  onto  which  the  video  image  produced  by  the  endoscope  may  be  mapped.  Figure  5  is  an 
example  of  an  endoscopic  image  after  proper  rectification  to  remove  lens-induced  distortions,  as  well  as 
those  due  to  the  arbitrary  shape  of  the  organ  surface,  that  has  been  accurately  mapped  onto  the  surface  of 
the  image  derived  from  a  3-D  CT  volume. 


Figure  5.  Endoscopic  images  from  a  tracked  endoscope,  mapped 
onto  the  surface  defined  by  a  3-D  digital  model  of  the  original 
object. 


4.  NON  NEUROLOGICAL  APPLICATIONS 


4.1  Orthopedics 

Computerized  image-guidance  of  spinal  and  orthopedic  procedures  has  lagged  somewhat  behind  its 
application  in  neurosurgery,  but  it  is  rapidly  growing  in  popularity.  Lavalee  et  al.43  discuss  a  computer- 
assisted  surgical  system  for  navigation  during  spinal  surgery,  using  CT  images  as  guidance.  Their  approach 
follows  closely  that  outlined  for  IGNS.  This  same  group44  describes  a  similar  technique  for  anterior  cruciate 
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ligament  reconstruction.  This  system  uses  a  workstation  and  a  three-dimensional  optical  localizer  to  display 
images  that  recreate  knee  kinematics.  In  the  future,  it  is  expected  that  standard  arthroscopy  will  be 
combined  with  a  tracking  system  and  “virtual  arthroscopy”  based  on  high  quality  MR  and/ or  CT  images  of 
the  joint  to  assist  in  such  procedures. 

Foley  et  al.4i  point  out  the  limitations  of  2-D  conventional  imaging  techniques  for  spinal  navigation,  and 
describe  a  three-dimensional  image-space  representation  of  the  surgical  volume,  using  a  specially  designed 
referencing  system  and  computer  workstation,  while  Nolte46  discusses  the  use  of  interactive  navigation  of 
surgical  instruments  for  the  fixation  of  spinal  implants.  The  use  of  rapid  3-D  MRI  was  investigated  by 
Martel  et  al. 47  to  acquire  images  suitable  for  image  guided  surgery  of  the  spine.  They  employ  a  very  fast  3-D 
MRI  sequence,  with  a  wide  bandwidth  and  short  echo  time  (TE)  to  minimize  susceptibility  distortions, 
along  with  MRI/CT  compatible  fiducial  landmarks.  This  permits  the  validation  of  the  MR  approach  against 
CT,  and  demonstrates  that  the  registration  can  be  undertaken  with  an  accuracy  of  0.4  mm  using  3-D  MRI. 
They  show  that  MR  can  effectively  be  as  accurate  as  CT  for  spinal  imaging  in  this  context,  and  conclude 
that  MRI  shows  promise  for  use  in  computer  assisted  surgery  of  the  spine. 

4.2  Breast 

Using  the  GE  intra-operative  magnet  discussed  above,  Gould  et  al? 8  demonstrated  that  interventional  MRI 
is  an  effective  tool  for  accurately  identifying  palpable  breast  tumors  and  guiding  their  surgical  excision.  The 
use  of  radiographic  and  ultrasonic  techniques  for  image-guided  breast  surgery  is  increasing,  but  is  currently 
restricted  to  fine  needle  aspiration  and  core  biopsy,  as  discussed  by  Staren  et  «/.49,50  They  comment  that 
future  care  of  patients  with  diseases  of  the  breast  will  continue  to  be  increasingly  dependent  on  image- 
guided  breast  biopsy  techniques.  This  approach  should  avoid  many  unnecessary  open  biopsies  for  benign 
lesions  and  facilitate  therapeutic  planning  for  malignant  lesions.  The  increasing  use  of  computer  assisted 
stereotactic  radiographic  and  ultrasound  breast  imaging  is  also  reviewed  by  Bums,51  who  suggests  that  such 
techniques  to  guide  percutaneous  core  sampling  of  the  abnormal  area  are  less  invasive,  less  painful,  highly 
accurate,  and  less  expensive  than  incisional  breast  biopsy  with  preoperative  needle  localization. 

4.3  Prostate 

Image  guidance  is  common  in  treatment  of  prostate  cancer,  but  its  application  is  focused  on  therapy  (via 
cryosurgery  or  brachytherapy)  rather  than  surgical  removal.  The  most  common  modality  in  use  for  image- 
guided  therapy  of  the  prostate  is  ultrasound,  and  the  recent  development  of  3-D  trans-rectal  ultrasound 
(TRUS)  imaging  techniques52'54  has  had  a  positive  impact  on  this  field.  When  employing  3-D  TRUS  for 
therapy,  a  series  of  2-D  US  images,  uniformly  spaced  in  angle,  are  reconstructed  to  form  a  3-D  volume.  This 
is  registered  with  a  therapeutic  probe  (radio-active  seed  implant  device  or  cryosurgery  probe)  and 
manipulated  in  much  the  same  manner  as  a  lesioning  device  is  introduced  into  the  brain.  Intraoperative  3-D 
TRUS  provides  unique,  pseudo  real-time  views  of  the  prostate,  which  facilitates  the  optimal  placement  of 
the  probes. 

4.4  Heart 

As  in  other  regions  of  the  body,  minimally  invasive  surgery  techniques  have  begun  to  play  an  increasing  role 
in  cardiac  surgery  in  an  attempt  to  reduce  the  significant  trauma  and  long  recovery  time  associated  with 
traditional  coronary  bypass  surgery.  Conventional  coronary  bypass  surgery  requires  a  sternotomy  and  a 
cardiopulmonary  bypass  procedure  that  subjects  patients  to  significant  trauma  and  lengthy  hospital  stays. 
Over  the  past  few  years,  minimally  invasive  direct  coronary  artery  bypass  (MIDCAB)  procedures  have  been 
introduced,  performed  via  instruments  introduced  into  the  chest  via  trochars  and  guided  endoscopically. 
Such  techniques  are  making  a  significant  impact  on  cardiac  surgery,  but  because  of  the  extreme  difficulty  in 
manipulating  the  instruments  at  the  distal  ends  of  the  trochars,  several  groups  have  recently  begun  to 
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perform  coronary  bypass  surgery  on  the  beating  heart  in  the  intact  chest  using  tele-operated  robots  inserted 
into  the  chest  via  intercostal  ports.55'57  The  use  of  robots  increases  dexterity  and  reduces  the  effect  of 
tremor  in  the  surgeon’s  hands. 

In  spite  of  the  sophistication  of  these  robotically  assisted 
systems,  the  use  of  images  in  the  planning  of  the  procedure  is 
presendy  limited  to  conventional  chest-radiographs  and 
angiograms.  The  use  of  such  simple  images  makes  it  extremely 
difficult  to  plan  the  positions  for  the  entry  ports  between  the 
ribs,  and  does  not  allow  for  any  level  of  guidance  during  the 
procedure.  The  operation  must  be  carried  out  solely  under  the 
guidance  of  a  1-2  cm  diameter  field-of-view  delivered  by  the 
inserted  endoscope.  Several  laboratories  have  recently 
extended  the  expertise  developed  in  neurosurgical  guidance  to 
build  a  virtual  cardiac  surgery  environment,  which  combines 
3-D  CT  and  4-D  (dynamic)  MRI  to  present  the  surgeon  with  a 
model  for  planning  and  guidance  during  the  procedure58,66 
Figure  6.  Simulated  environment  for  (Figure  6).  Initially,  these  efforts  have  been  based  on  static 

robotically-assisted  coronary  bypass  surgery,  cardiac  images  aimed  at  planning  appropriate  inter-costal  port 

showing  positions  of  robotically-controlled  locations  and  robot  orientations.  This  work  is  however  moving 

surgical  ports  endoscope.  rapidly  in  the  direction  of  virtual  dynamic  surgical  simulation, 

planning  and  guidance  environments,  that  will  synchronize  the 
dynamics  of  the  modeled  cardiac  environment  with  that  of  the  beating  heart. 


5.  COORDINATED  APPLICATION  DEVELOPMENT 

Many  groups  across  the  globe  are  working  on  problems  in  image-guided  surgery,  and  many  different 
software  and  hardware  platforms  have  been  employed.  This  situation  not  only  makes  it  difficult  to  share 
resources  across  the  community,  it  also  raises  the  problem  of  legacy  software  being  constrained  to  a 
particular  (outdated)  hardware  platform.  Nevertheless,  over  the  past  few  years,  many  groups  have  begun  to 
use  the  rapidly  evolving  VTkh  environment  for  developing  medical  imaging  applications.  Working  with  such 
a  suite  of  robust  tools  has  significant  advantages  producing  increased  development  speed,  reliability,  and 
reusability  of  code.  In  addition,  its  open  source  nature  encourages  contribution  from  the  user  community. 
The  development  of  new  applications  in  medical  image  visualization  and  surgical  planning  requires  the 
completion  of  many  common  tasks  such  as  reading,  re-sampling  and  segmenting  images,  followed  by 
visualization  via  surface  or  volume  rendering,  etc.  The  intra-operative  use  of  such  tools  requires  that  images 
be  registered  to  the  patient,  and  that  tools  and  probes  are  tracked.  For  such  systems  to  be  employed  by 
someone  other  than  the  original  author,  the  user  must  be  provided  with  basic,  easy  to  understand  user 
interface  components.  Computer  and  end-application  hardware  changes  rapidly,  as  do  operating  systems 
and  network  environments.  Such  factors  emphasize  the  advantages  of  having  access  to  an  independent 
collection  of  reusable  software  components  that  are  hardware  and  operating  system  independent,  and  which 
can  be  assembled  rapidly  to  prototype  new  applications. 

h  http://www.kitware.com 
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In  our  lab,  we  have  developed  a  set  of  such  components  that  address  some  of  the  above-mentioned 
concerns.59  They  are  written  in  both  C++  (some  as  VTk  classes)  and  Python,  but  all  are  accessible  from 
Python,  a  byte  compiled  scripting  language  that  is  gaining  rapid  acceptance.  Applications  built  using  these 
components  have  been  used  on  the  Red  Hat  Linux,  Silicon  Graphics  Irix,  Microsoft  Windows,  and  Mac  OS 
X  platforms.  Rigorous  object  oriented  software  design  methods  have  been  applied  to  ensure  hardware 
independence  and  a  standard  application  programming  interface  (API).  There  are  components  to  acquire, 
display  and  register  images  from  MRI,  MRA,  CT,  Computed  Rotational  Angiography  (CRA),  Digital 
Subtraction  Angiography  (DSA),  2D  and  3D  ultrasound,  video  and  physiological  recordings.  We  have  also 
implemented  interfaces  to  various  tracking  systems  for  intra-operative  use.  Over  the  past  two  years,  these 
components  have  been  used  to  create  general  image  manipulation  and  viewing  tools,  a  deep  brain 
functional  atlas31  (Figure  5),  a  3D  ultrasound  acquisition  and  display  platform24,  a  tracked  neuroendoscope 
guidance  system61  (Figure  6),  a  prototype  minimally  invasive  robotic  coronary  artery  bypass  graft  planning 
system58  (Figure  7),  and  a  frame-based  stereotaxy  neurosurgery  planning  tool59.  We  believe  that  by  following 
this  approach,  new  applications  can  be  developed  more  rapidly,  they  can  be  readily  tailored  towards  the 
needs  of  users,  and  the  transition  from  a  research  to  a  clinical/ commercial  environment  can  be  facilitated. 
Further  details  of  this  approach  can  be  found  in  Starreveld  et  al. 59 

6.  CHALLENGES  IN  IMAGE-GUIDED  SURGERY 

The  current  emphasis  on  the  use  of  minimally  invasive  therapies  in  all  parts  of  the  body  has  made  image- 
guidance  for  therapy  and  surgery  a  rich  area  for  research  and  development.  Many  of  these  applications  present 
fascinating  problems,  and  one  of  the  most  challenging  is  in  the  area  of  minimally  invasive  robotically-assisted 
cardiac  surgery,  where  therapy  is  performed  on  the  beating  heart  through  several  small  holes  between  the  ribs. 
In  order  to  adequately  provide  Full  intra-operative  image-guidance  to  the  surgeon  during  the  procedure,  not 
only  must  pre-operative  dynamic  images  be  matched  to  the  intra-operative  anatomy  of  the  patient  but  proper 
account  must  be  taken  of  the  complex  relative  motion  undergone  by  the  heart  and  thorax  during  the 
procedure.  Currently,  the  efficacy  of  this  procedure  is  limited  by  the  lack  of  sophistication  of  the  robotic  end- 
effectors,  allowing  only  single  accessible  coronary  vessels  to  be  bypassed  using  this  procedure.  Improvements 
to  the  dexterity  of  these  robots  will  enable  more  complex  procedures  to  be  performed  using  these  minimally- 
invasive  approaches. 

Factors  currently  limiting  the  wide  acceptance  of  image-guided  surgery  include  the  lack  of  universal  image 
standards,  less  than  intuitive  user  interfaces,  differences  in  image-registration  protocols,  incompatibility  of 
surgical  tools  with  guidance  systems,  lack  of  adaptation  of  systems  to  user  preferences,  and  inherent  errors  in 
imaging  and  tracking  systems.  While  it  is  true  that  the  introduction  of  ACR-NEMA  followed  by  DICOM-3 
has  gone  a  long  way  towards  the  creation  of  a  universal  image  format  standard,  there  nevertheless  remain 
many  DICOM  “flavors”.  In  addition,  DICOM  does  not  directly  support  3  and  4-dimensional  datasets  or  3-D 
objects.  Images  are  still  treated  as  a  serried  as  “byte-slices”  rather  than  as  samples  of  some  parameter  within  a 
multi-dimensional  space.  Some  of  these  limitations  have  been  addressed  by  the  MINC  (Medical  Image  Net- 
CDF)  format,  based  on  Net-CDF  and  extended  at  the  Montreal  Neurological  Institute  by  Neelin  et  a.6M 
MINC  adopted  by  many  groups  in  the  neuroscience  imaging  community  and  beyond  as  an  effective 
format /environment  for  medical  imaging  reseach.  The  need  to  evolve  coherent  user  interface  and 
interconnectivity  standards  that  could  be  adopted  by  multiple  manufacturers  was  expressed  by  Bucholz  in  his 
recent  keynote  address  at  the  MICCAI-2000  meeting.  To  this  end  he  has  proposed  the  SurgON  standard', 


d  The  MINC  Format.  MNI  Website  .http://www.bic.mni.mcgill.ca/software/minc/minc.html 

e  httn:.7Thalamus.  slu.edu/SurgQN/ 


48 


A  View  of  Image-Guided  Therapeutic  Procedures,  Surgical  Simulation  and  Medical  Robotics 


that,  if  adopted  universally,  could  solve  many  of  the  inter-connectivity  and  user  interface  problems  that  exist 
with  current  systems. 


7.  SIMULATION 

The  development  of  tools  for  image-guided  surgery  and  therapy  has  led  to  the  need  for  an  environment  to 
accurately  represent  the  patient  using  computer  methods,  to  simulate  surgical  and  therapeutic  procedures  in 
as  realistic  a  manner  as  possible,  without  involving  human  subjects.  This  approach  becomes  particularly 
important  as  surgical  procedures  become  minimally  invasive  and  robotically  assisted.  The  need  for  accurate 
simulation  in  this  environment  can  be  likened  to  the  need  for  simulation  in  the  aviation  industry,  i.e. 
exposing  the  pilot/ surgeon  to  realistic  scenarios  and  monitoring  their  responses. 

Simulation  fulfills  the  needs  to  compare  alternative  procedures,  develop  and  refine  interventions,  reduce 
complications,  increase  predictability,  train  practitioners,  customize  instruments  and  devices,  and  improve 
outcomes.  The  input  data  for  such  a  simulator  are  acquired  from  multiple  imaging  modalities,  including  CT, 
ultrasound,  and  MRI,  and  can  be  in  the  form  of  pre-,  intra-  and  post-operative  three-dimensional  images. 
In  reality,  simulation  is  part  of  the  continuous  spectrum  that  already  exists  for  image-guided  surgery.  The 
level  of  realism  in  the  presentation  of  the  data  allows  us  to  progress  from  simple  guidance  of  the  procedure 
to  realistic  simulation.  Models  of  the  skeleton  and  soft  tissue  may  now  be  synthesized  from  volumetric 
images,  and  visualized  and  modified  interactively.  Individualized  electronic  regional  body  atlases  may  also  be 
created  using  deformable  templates  or  by  applying  image  fusion/registration  methods.  Once  modeled  and 
visualized,  simulation  is  accomplished  with  various  interactive  devices,  both  visual  and  tactile,  that  mimic 
biopsy,  minimally  invasive,  and  open  surgical  and  therapeutic  procedures,  tele-surgery,  intraoperative 
navigation,  virtual  endoscopy,  gene  therapy,  and  can  provide  the  necessary  training.  Glombitza  et  al.62 
recently  presented  an  overview  of  the  state  of  the  art  of  virtual  surgery,  citing  its  use  in  a  variety  of 
environments. 

An  important  aspect  of  realistic  simulation  is  the  force  feedback  sensed  by  the  operator  conveying  realism  at 
the  tactile  level.  This  probably  represents  the  most  difficult  aspect  of  the  simulation  cycle.  While 
sophisticated  multi-degree  of  freedom  force-feedback  (haptic)  devices  exist  (Sensable  Technologies  Incf, 
MPB  Technologies8),  they  are  limited  to  applying  linear  or  rotational  force  feedback  to  the  operator,  are  not 
able  to  provide  textural  information  (to  the  fingertips  for  example),  and  may  be  limited  with  respect  to  the 
rate  at  which  they  can  respond  to  input.  It  is  likely  that  in  some  instances,  physical  models  (registered 
spatially  with  the  virtual  model)  could  be  employed  to  provide  the  appropriate  haptic  feedback  when 
performing  such  procedures  as  cutting  and  probing  tissues.  An  example  might  be  an  Agar  or  Poly-vinyl 
alcohol  cryogel63  phantom  that  has  been  manufactured  to  simulate  the  mechanical  properties  of  the  tissue 
being  modeled.  If  the  underlying  finite  element  model  that  is  driving  the  mechanical  simulation  and 
visualization  is  sufficiently  sophisticated  to  mimic  the  behavior  of  the  phantom  (when  cut  for  example),  the 
physical  model  could  provide  much  of  the  necessary  mechanical  feedback  to  the  operator. 

8.  FUTURE  DIRECTIONS 

Although  image-guided  procedures  are  commonplace  in  neurosurgery,  there  have  been  few  efforts  to 
quantify  their  effectiveness,  either  in  terms  of  patient  outcome  or  cost  savings.  One  study  that  has 
attempted  to  do  so  was  presented  recently  by  Paleologos  et  al.,6A  who  demonstrate  some  benefit  (in  terms 


f  http://www.sensable.com/products/6dof.htm 
8  http://www.mpb-technologies.ca/space/p_ffeedom6s.html 
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of  reduced  complication  rates,  shorter  hospital  stays)  through  the  use  of  image -guidance  for  meningioma 
surgery  via  craniotomy  vs.  conventional  approaches.  They  point  out  that  it  is  difficult  to  construct  robust 
studies  to  validate  such  procedures,  and  that  many  of  the  advantages  relate  to  the  increased  comfort  of  the 
surgeon  performing  the  procedure.  While  difficult  to  generate  from  this  small  study,  they  acknowledge  the 
additional  benefits  of  integrating  multi-modality  imaging  and  correcting  for  intra-operative  brain  shift  would 
be  expected  to  significandy  enhance  the  acceptance  of  such  techniques.  While  image-guide  surgery  systems 
are  becoming  increasingly  sophisticated  perhaps  the  greatest  challenge  to  the  successful  adoption  of  image- 
guided  surgery  systems  in  the  operating  room  of  the  future,  relates  not  to  the  technology',  but  to  the  user 
interface  and  the  acceptance  of  the  technique.  Over  the  last  few  years,  inexpensive  high-speed  computing 
hardware  has  become  available  to  allow  interactive  surgical  image-guidance  to  be  performed  in  real  time, 
with  the  images  presented  to  the  surgeon  in  a  “virtual-reality”  format.  However,  little  effort  has  been 
expended  to  date  on  ensuring  that  the  human  factors  issues  relating  to  the  use  of  such  equipment  in  the  OR 
have  been  adequately  addressed.  Visarius  et  a/.65  point  out  that  the  majority  of  image-guided  surgical  systems 
in  clinical  use  still  require  the  involvement  of  a  “systems  engineer”  to  ensure  that  the  procedure  runs 
smoothly.  It  is  incumbent  on  all  who  develop  such  systems  that  they  focus  on  reducing  the  cost  of  the 
equipment,  making  it  available  on  demand,  and  designing  it  so  that  it  can  be  used  by  OR  staff  as  a  routine 
item  of  surgical  equipment.  Patient  images  must  be  automatically  available  in  the  OR  in  the  appropriate 
format;  patient-image  registration  steps  must  be  trivial  and  robust,  all  tools  should  be  recognized  and 
trackable  by  the  system,  and  images  need  to  integrated  with  the  actual  patient  view  so  that  the  surgeon  is  not 
constandy  changing  his  visual  field.  It  will  only  be  when  such  systems  are  unobtrusive,  provide  simple 
access  to  the  image  sources  and  can  be  operated  in  an  intuitive  manner  by  traditional  operating  room  staff, 
that  they  will  be  employed  routinely  in  the  OR. 

Beyond  neurosurgery,  minimally-invasive  image-guided  techniques  will  continue  to  play  increasingly 
important  roles  in  cardiac  surgery,  breast,  orthopedic  surgery,  prostate  therapy,  and  brachytherapy  of 
various  organs.  The  use  of  tele-robotic  control  will  increase  the  precision  of  these  procedures  through 
tremor  control  and  dexterity  enhancement,  and  permit  more  procedures  to  be  performed  under  direct  image 
guidance  (in  conventional  MR  scanners  or  high  dose-rate  x-ray  fluoroscopy).  In  adition,  many  of  the 
techniques  discussed  here  are  finding  application  in  conventional  external  beam  radiation  therapy. 

9.  CONCLUSION 

Image-guided  approaches  have  evolved  significandy  over  the  past  20  years  and  stand  to  make  further 
important  contributions  to  a  variety  of  surgical  and  therapeutic  procedures,  rendering  them  less  invasive  and 
less  traumatic  to  the  patient.  There  are  many  challenges  in  the  path  of  the  general  acceptance  of  these 
techniques  by  the  surgical  community.  In  addition  to  the  general  lack  of  quantifiable  evidence  regarding  the 
demonstrable  efficacy  of  these  approaches,  there  is  a  natural  reluctance  of  many  practitioners  to  embrace 
technology-based  approaches,  particularly  when  faced  with  the  prospect  of  having  to  leam  on  a  new 
paradigm,  or  to  place  reliance  on  an  additional  level  of  “engineering”  support  during  the  surgical  procedure. 
For  this  reason,  the  key  developments  in  this  field  must  include  the  provision  of  a  largely  automated  and 
“bullet-proof’  infrastructure,  where  image-management  becomes  a  trivial  operation,  the  delivery  of  user 
interfaces  that  are  intuitive  and  non-threatening  and  the  ability  to  make  systems  available  to  the  surgeon  on 
demand  with  a  minimum  of  additional  planning. 
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Outline  of  Surgical  Simulation 

Kevin  Cleary  and  Robert  L.  Galloway  Jr. 


Medical  Simulators  are  interactive  virtual  environments  used  to  improve  human  performance  of 
medical  tasks.  While  they  need  not  be  computer-based  the  focus  for  this  paper  is  personal 
computer-based  systems.  The  uses  of  surgical  simulators  are  many.  First  is  medical  education  and  or 
training.  The  population  here  can  range  from  experienced  physicians  learning  new  procedures  to 
nurses,  medical  students,  military  medical  corpsmen,  emergency  medical  personnel  and  even  to  the 
general  public  for  first  aid  procedures.  Simulators  can  also  be  used  for  scientific  analyses  such  as 
biomechanical  studies  or  device  design  and  testing.  With  the  advent  of  the  image-guided 
procedures  described  elsewhere  in  this  document,  it  should  be  apparent  that  simulators  could  allow 
for  more  realistic  preprocedural  treatment  planning. 

Types  of  Medical  Simulators 

There  are  several  types  of  surgical  simulators.  The  list  includes:  plastic  mannequins,  computer  based 
mannequins  and  virtual  environment  simulators.  While  each  has  their  own  uses  there  is  a  growing 
convergence  of  system.  However,  for  simplicity’s  sake  they  can  be  grouped  as  below: 

Plastic  mannequins  -  Physical  models  for  practicing  procedures  such  as  intubation  and  IV 
placement.  Examples  are  the  patient  simulators  MedSim  and  METI.  They  have  computerized 
sounds  and  physiology  and  are  used  for  skill  assessment 
and  trauma  training  applications. 

An  example  of  a  plastic  mannequin 
is  shown  in  Figure  1 . 


Figure  1. 


The  second  class  of  medical  simulators  are  computer-based  mannequins.  These  simulators  are  used 
for  training  and  testing  skills  in  anesthesiology  and  related  applications.  The  information  is  provided 
by  visual  feedback  on  a  computer  system  of  anatomic  structures  and  physiologic  signals 
corresponding  to  the  users  input  parameters.  An  example  of  a  computer  based  mannequin  is 
shown  in  Figure  2. 
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An  example  of  a  computer-based 
mannequin  is  the  CathSim  Needle 
Insertion  Simulator  from 
Immersion  Medical.  Here  a  tactile 
interface  with  a  physics-based  simulation  is 
provided.  The  role  of  this  simulator  is  to 
provide  assessment  of  skills  in  patient  case 
studies.  By  having  such  a  controlled 
environment,  studies  can  be  performed 
across  groups  of  users  and  longitudinally  in 
users  to  measure  changes  in  skills. 


Figure  2 


The  final  major  class  of  medical 
simulators  are  Virtual  environment 
(VE)  simulators  -  Contain  a 
computer  graphics  model  and 
provide  an  immersive  experience. 
One  example  of  this  class  on 
simulator  is  the  Virtual  Surgery 
Simulator  from  Boston  Dynamics, 
Inc.  (shown  in  Figure  3).  Here  a 
Mirror-based  workbench  and 
Stereoscopic  glasses  place  the  image 
in  the  physicians  natural  visual  field. 
Feedback  is  provided  both  with  the 
haptic  PHANToM  interface  and 
audio  information. 


Figure  3 


Example  of  Medical  Simulation  System 

A  joint  effort  between  Georgetown  University  and  the  Korean  Advanced  Institute  of  Science  and 
Technology  (KAIST)  has  been  developing  a  spine  biopsy  simulator.  Here  their  concern  is  for  a 
common  medical  procedure  which  has  a  well  described  physiological  structure.  The  relatively 
simple  anatomy  and  interventional  nature  of  the  procedure  make  this  application  well-suited  to 
simulation. 
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Figure  4 

The  Spine  Biopsy  simulator  is  shown  in  the  diagram  in  Figure  4.  Individual  components  of  that 
device  include  the  user  interface  shown  in  Figure  5. 


Figure  5 


GUI  developed 
by  the  Image 
Systems  Lab, 
Dept  of 
Electrical 
Engineering, 
KAIST,  Korea, 
Directed  by  Dr. 
Jong  Beom  Ra 
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Figure  6. 


As  shown  in  Figure  6  the  GUI  can  provide  axial  views  of  the  simulator’s  “progress”  into  the  tissue 
as  well  as  three-dimensional  renderings  such  as  the  one  shown  in  Figure  7.  Since  the  screen  is  two 
dimensional  and  the  task  is  three  dimensional ,  multiple  image  types  are  needed  to  provide  the 
desired  guidance. 
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Beyond  the  matching  of  physical  and  image  position,  the  goal  of  a  simulator  is  to  provide  a  “feel”  to 
the  user.  Therefore  it  is  desirable  to  provide  accurate  real-time  replication  of  interaction  between  an 
instrument  &  soft  tissue.  For  the  spinal  procedures  there  are  classes  of  anatomical  models:  non- 
deformable,  deformable  surface-based  and  deformable  volume-based.  The  force-feedback  methods 
include  a  tissue/force  profile  scheme,  surface  rendered  modeling  and  finite  element  analysis. 

For  the  spine  procedures  a  heuristic  model  consisting  of  3  regions  was  developed.: 

1 .  Initial  needle  puncture  (fat  to  muscle) 

2.  Homogeneous  force  (muscle) 

3.  Hard  stop  (bony  structure) 

These  zones  are  shown  in  Figure  8. 
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Figure  8. 


Fat 

Muscle 

Bone 


The  physical  interface  for  the  simulator  is  shown  in  figure  9.  In  this  device  the  position  of  the  probe 
is  tracked  and  force  feedback  appropriate  to  the  zone  of  application  is  provided  by  the  PHANToM 
haptic  device. 


Spine 

Biopsy 

Simula 

tor 

Model 


Figure  9 
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Figure  10 


Conclusions 

Surgical  or  procedural  simulators  can  provide  standardized  training  for  specific  procedures. 
They  are  flexible,  allowing  reprogramming  to  represent  a  variety  of  clinical  presentations.  While  not 
yet  ready  to  replace  cadaveric  training  for  all  procedures,  simulators  can  be  an  important  adjunct 
with  applications  growing  with  the  sophistication  of  the  simulators. 
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ABSTRACT 

As  a  demonstration  platform,  we  are  developing  a  robotic  biopsy  testbed  incorporating  a  mobile  CT 
scanner,  a  small  “needle  driver”  robot,  and  an  optical  localizer.  This  testbed  will  be  used  to  compare 
robotically  assisted  biopsy  to  the  current  manual  technique,  and  allow  us  to  investigate  software 
architectures  for  integrating  multiple  medical  devices.  This  is  a  collaboration  between  engineers  and 
physicians  from  three  universities  and  a  commercial  vendor.  In  this  paper  we  describe  the  CT-directed 
biopsy  technique,  review  some  other  biopsy  systems  including  passive  and  semi-autonomous  devices, 
describe  our  testbed  components,  and  present  our  software  architecture.  This  testbed  is  a  first  step  in 
developing  the  image-guided,  robotically  assisted,  physician  directed,  biopsy  systems  of  the  future. 

Keywords:  biopsy,  medical  robotics,  software  architecture 


1.  INTRODUCTION  AND  OVERALL  CONCEPT 


I  iQpr  Intprfarp  .^nftwarp 


Biopsy  is  a  common  procedure  in  the  medical  field. 
While  in  most  cases  this  procedure  can  be 
completed  without  difficulty,  there  are  limitations  to 
the  accuracy  obtainable  using  freehand  techniques. 
In  addition,  CT-directed  biopsy  can  be  tedious  and 
time  consuming,  since  frequent  re-imaging  may  be 
required. 

For  these  reasons,  we  are  developing  a  robotic 
biopsy  testbed  incorporating  a  mobile  CT  scanner,  a 
small  “needle  driver”  robot,  and  an  optical  localizer. 
A  system  diagram  is  shown  in  Figure  1 


Figure  1.  Testbed  components 
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The  goals  of  the  testbed  are  to: 

1 .  Develop  a  demonstration  system  for  robotically  assisted  biopsy 

2.  Compare  robotically  assisted  biopsy  to  the  current  manual  technique 

3.  Serve  as  a  testbed  for  investigating  software  architectures  for  incorporating  multiple  medical  devices 
This  testbed  is  part  of  a  collaboration  between  Georgetown  and  the  Urology  Robotics  Laboratory 
(URobotics  Lab)  of  the  Johns  Hopkins  Medical  Institutions  (http:  /  /urology. jhu.edu/ urobotics/)  and  the 
Computer  Integrated  Surgical  Systems  and  Technology  Center  (http://cisstweb.cs.ihu.edu/web/)  centered 
at  the  Johns  Hopkins  University.  As  part  of  this  collaboration,  we  are  also  planning  to  apply  the  needle 
driver  robot  to  percutaneous  spine  procedures  such  as  nerve  and  facet  blocks  [1]. 

The  remainder  of  the  paper  is  organized  as  follows.  The  biopsy  task  is  described  in  Section  2.  In  Section  3, 
related  devices  for  mechanically  guided  biopsy  are  reviewed.  The  system  components  for  our  testbed  are 
described  in  Section  4.  The  software  design  is  presented  in  Section  5. 

2.  BIOPSY  PROCEDURE 

This  procedure  requires  a  computed  tomography  (CT)  scanner  and  trained  technologist;  a  special  biopsy 
needle;  and  a  “biopsy  tray”  witii  appropriate  syringes,  needles,  anesthetic  solution,  and  sterile  towels  [2,  3]. 

It  should  not  be  performed  (unless  absolutely  necessary)  on  patients  with  bleeding  disorders  or  otherwise  at 
high  risk  for  hemorrhage. 

Each  patient  receives  a  pre-procedure  CT  scan  to  ascertain  the  lesion  site,  and  to  determine  the  safest  route 
of  approach  to  the  lesion.  Once  the  target  and  the  skin  entry  point  are  chosen,  the  skin  site  is  marked  with  a 
radiopaque  label  (such  as  a  BB  or  a  small  needle  taped  to  the  skin).  An  additional  axial  CT  image  of  this  site 
is  then  obtained  to  confirm  the  coordinates,  and  calculate  the  desired  distance  from  skin  to  target.  The 
chosen  trajectory  should  avoid  (if  possible)  approaching  pleura,  peritoneum,  or  major  vessels  or  nerves. 

The  entry  site  is  then  prepped  with  a  sterile  skin  cleaning  agent  (such  as  povidone  iodine  solution),  and 
draped  with  sterile  towels.  Local  anesthetic  (lidocaine  HC1 1%  and/or  bupivacaine  HC1  0.25-0.5%)  is 
infiltrated  into  the  entry  site  using  a  5  or  10  cc  syringe  with  a  25-  or  30-gauge  needle.  After  initial  superficial 
anesthesia  is  achieved,  the  anesthetic  solution  may  be  injected  deeper  along  the  proposed  biopsy  track  using 
a  longer,  slightly  wider  needle  (such  as  a  16-  to  22-  gauge  spinal  needle).  In  spine  biopsy  procedures,  care  is 
taken  to  anesthetize  down  to  and  including  the  periosteum. 

At  this  point,  a  spinal  needle  is  inserted  partially  along  the  biopsy  track,  and  a  CT  image  is  taken  to  confirm 
proper  site  and  trajectory.  If  unsatisfactory,  the  spinal  needle  is  repositioned,  and  additional  images 
obtained.  If  adequate,  the  needle  is  advanced  the  rest  of  the  way,  and  target  acquisition  is  confirmed  with 
another  CT  image.  If  the  patient  reports  radicular  pain  during  the  needle  placement,  the  needle  is 
redirected;  if  further  attempts  also  elicit  pain,  a  new  entry  site,  trajectory,  and/or  target  may  need  to  be 
selected. 

Once  a  satisfactory  angle  of  approach  is  confirmed,  the  spinal  needle  is  removed,  and  the  larger  biopsy 
needle  is  carefully  inserted  along  the  same  tissue  path.  A  final  CT  image  is  obtained  to  confirm  that  the 
needle  tip  is  in  the  target  tissue  before  any  samples  are  taken.  Pressure  is  applied  to  the  biopsy  needle,  along 
with  a  twisting  or  cutting  motion  (depending  on  the  type  of  biopsy  needle  used;  the  recommended 
technique  is  described  in  the  manufacturer’s  instructions).  Before  the  core  of  tissue  is  removed,  another  CT 
slice  is  taken. 
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While  one  tissue  core  may  suffice,  many  investigators  take  two  or  three  samples  to  help  ensure  an  adequate 
yield.  Some  may  choose  to  have  a  surgical  pathologist  or  cytologist  on  hand  to  examine  the  tissue  specimen 
for  suitability. 

Once  enough  tissue  has  been  obtained,  some  investigators  obtain  one  last  CT  image  following  needle 
removal  to  demonstrate  the  biopsy  defect  in  the  target  tissue.  Direct  pressure  may  be  held  on  the  skin  entry 
site  for  several  minutes  to  aid  hemostasis,  if  needed. 

3.  LITERATURE  REVIEW:  MECHANICALLY  GUIDED  BIOPSY 

Other  researchers  have  developed  robotic  systems  to  aid  in  biopsy  tasks.  These  include  passive  positioning 
systems,  which  provide  image  guidance  to  assist  the  physician  in  orienting  the  biopsy,  as  well  as  semi- 
autonomous  robots  which  position,  drive,  and  guide  the  biopsy  needle  under  remote  physician  control. 

The  PinPoint™  system,  developed  by  Marconi  Medical  Systems  (Cleveland,  OH,  formerly  Picker),  is 
representative  of  the  passive  robotic  biopsy  assistant  type  [4].  PinPoint  is  a  frameless  stereotactic  arm  for 
use  in  planning  CT-guided  biopsy  (Figure  2a).  The  arm  is  direction  encoded  so  that  the  intervention  path 
can  be  visualized  on  CT  and  evaluated.  Once  the  optimum  biopsy  needle  path  is  determined,  the  Pinpoint 
arm  can  be  locked  into  place  to  serve  as  a  stationary  guide  to  needle  placement  (Figure  2b). 


Figure  2a.  Pinpoint™ 
arm  (upper  left)  and 
CT/fluoroscopy  system 


Figure  2b.  Pinpoint™ 
arm  locked  in  place  to 
serve  as  a  needle  guide 


(Courtesy  of  Marconi  Medical  Systems: 

www.picker.com/ct/ctro/ctrovenue.htTnl~) 


Semi-autonomous  systems  have  been  developed  for  use  in  breast  and  brain  tissue  biopsy.  The 
Mammotome®,  manufactured  by  Ethicon  Endo  Inc  (Cincinnati,  OH)  a  Johnson  and  Johnson  Company, 
and  the  Advanced  Breast  Biopsy  Instrumentation  (ABBI®)  system,  a  product  of  the  United  States  Surgical 
Corporation  (Norwalk,  CT),  are  two  widely  available  breast  biopsy  systems.  Both  systems  allow  remote, 
image-guided  placement  and  manipulation  of  the  biopsy  tool. 

The  Mammotome  breast  biopsy  system  is  composed  of  a  thin  probe  attached  to  a  motorized  unit  and  an 
integrated  vacuum  source  [5].  Under  local  anesthesia,  the  biopsy  probe  is  introduced  through  a  W’inch 
mcision.  The  probe  is  navigated  under  image  guidance  to  the  target  selected  by  the  physician.  Both  CT  and 
ultrasound  guidance  are  compatible  with  the  Mammotome.  Once  in  position,  a  vacuum  system  draws  tissue 
into  the  probe  core  and  harvests  small  samples.  The  operator  can  navigate  the  probe  to  more  than  one 
target  per  insertion,  allowing  wide  sampling  from  only  one  access  point  [6]. 


The  ABBI  system,  allows  stereotactically  guided,  mechanically  driven  fine  needle  biopsy  and  a  novel  “core 
needle”  breast  biopsy  [7].  Rather  than  harvesting  microhistological  specimens  from  a  variety  of  locations  in 
the  breast,  the  “core  needle”  breast  biopsy  option  removes  a  single,  solid  specimen,  which  can  vary  in  size 
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from  5  mm  to  20  mm,  depending  on  physician  preference.  There  is  some  evidence  that  one-piece  cytologic 
specimens  can  aid  in  the  diagnosis  of  certain  breast  disorders,  an  advantage  of  the  ABBI  core  needle  biopsy. 
Studies  comparing  the  effectiveness  of  the  ABBI  core  needle  biopsy  to  the  Mammotome’s  microhistological 
approach  are  ongoing  [8].  Because  the  ABBI  core  needle  system  has  the  potential  to  remove  large  pieces  of 
target  tissue  under  physician  guidance,  it  has  potential  as  a  treatment  modality  in  addition  to  a  biopsy  tool. 
This  application  is  still  under  investigation. 

Two  representative  semi-autonomous  neurosurgical  biopsy  systems  include  the  Minerva,  developed  by  the 
Laboratory  of  Microengineering  at  the  Swiss  Federal  Institute  of  Technology  in  Lausanne,  Switzerland,  and 
the  NeuroMate™,  manufactured  by  Integrated  Surgical  Systems  (Davis,  CA). 

Minerva  is  a  CT  guided,  multi- function  neurosurgical  robot  [9].  It  operates  inside  a  CT  gantry  with  free 
longitudinal  movement  allowing  cranial  scans  at  any  level.  Under  the  physician  s  remote  control,  the 
Minerva  robot  can  manipulate  two  stereotactic  instruments  in  addition  to  the  tool  for  automatic  penetration 
of  the  skin,  skull,  and  meninges.  This  allows  the  Minerva  to  perform  a  complete  stereotactic  procedure 
without  physical  intervention  by  the  physician.  In  addition  to  biopsy,  this  system  has  been  used  for 
deposition  of  living  encapsulated  cells,  electrode  implantation,  placement  of  radioactive  sources, 
electrostimulation,  and  tissue  aspiration  [10]. 

The  NeuroMate  stereotactic  surgical  system  was  originally  developed  at  the  University  of  Grenoble,  France 
[11,  12]  and  is  now  available  from  ISS.  NeuroMate  is  an  image-directed  robotic  assistant  for  frameless 
stereotactic  neurosurgical  applications  (Figure  3).  It  can  function  to  orient  and  position  surgical  instruments 
under  image  guidance.  Much  like  Minerva,  it  is  also  capable  of  carrying  out  surgical  procedures  under 
remote  physician  control. 


Figure  3.  NeuroMate™  robotic  assistant  for  neurosurgical  stereotactic  applications 

(Courtesy  of  Integrated  Surgical  Systems:  wwvv.robodoc.com/products/neuroinate.html') 


4.  SYSTEM  DESCRIPTION  AND  COMPONENTS 

The  testbed  components  (Figure  1)  are  described  in  the  following  sub-sections 

4.1  Mobile  CT  Scanner 

The  Tomoscan  M  is  a  mobile  CT  scanner  (Philips  Medical  Systems,  Eindhoven,  Netherlands)  that  is  easily 
transportable  within  the  hospital.  The  system  has  three  components  including  a  gantry,  CT  table,  and 
operator’s  workstation.  The  gantry  aperture  is  60  cm  with  a  maximal  field  of  view  of  460  mm.  Both  the 
gantry  and  the  CT  table  can  translate,  35  cm  and  150  cm  respectively.  The  images  have  a  resolution  of  512 
by  512  pixels  and  can  be  transferred  to  other  systems  using  the  digital  imaging  and  communications  in 
medicine  (DICOM)  standard.  Protocols  for  cervical,  thoracic,  and  lumbar  spine  exist  with  slice  thickness 
options  of  2,  3,  5,  and  10  mm.  The  system  has  a  tube  voltage  of  130  kV  and  uses  a  relatively  low  tube 
current  between  10  and  50  mA,  thereby  minimizing  dose  exposure. 
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4.2  The  Robotic  System 

The  robotic  system  will  be  based  on  the  PAKY-RCM  (Percutaneous  Access  to  the  KidneY  -  Remote 
Center  of  Motion  )  robot  that  has  been  initially  developed  at  Johns  Hopkins  for  percutaneous  access  of  the 
renal  collecting  system  [13,  14].  The  robot,  schematically  represented  in  Figure  4,  consists  of  a  passive 
positioning  and  supporting  arm  (The  GREY  Arm)  [15],  an  active  remote  center  of  motion  orientation 
mechanism  (RCM),  and  a  radiolucent  needle  driver  (PAKY).  The  device  will  be  mounted  over  the  CT  table 
using  a  bridge  fixture  as  depicted  in  Figure  4. 


Figure  4.  PAKY-RCM  robot  and  bridge  fixture 


The  overall  system  comprises  eleven  degrees  of  freedom  (DOF).  The  first  eight  DOF  are  used  for  the  initial 
positioning  of  the  robot  in  close  proximity  of  the  skin  insertion  site  and  firmly  locked  during  the  operation. 
The  remaining  three  DOF,  implemented  by  the  RCM  robot  and  PAKY  needle  driver,  are  sufficient  for 
orienting  and  inserting  the  needle  at  the  desired  target  through  the  preset  skin  insertion  point.  The  main 
advantage  of  this  minimal  kinematic  architecture  is  the  inherent  safety  given  by  the  restricted  mobility  of  the 
mechanical  components.  Moreover,  separating  the  kinematics  of  the  orientation  and  needle  insertion  yields 
decoupled  needle  motion  capabilities,  thus  improving  safety. 

The  needle  driver  is  constructed  of  acrylic  plastic  which  is  radiolucent  and  easy  to  manufacture  as  a  sterile 
disposable  part.  Driver  radiolucency  is  essential  in  image-guided  procedures  for  providing  unimpeded  target 
visualization.  Whereas  the  driver  is  sterilized,  in  clinical  use  the  additional  components  of  the  system  in  close 
proximity  of  the  operation  site  are  covered  with  a  sterile  bag. 

The  robot  accommodates  joystick  control  for  simple  maneuvers  and  full  computer  control  for  the  actual 
image-guided  procedure.  The  electronic  circuitry  will  be  fully  enclosed  in  the  supporting  bridge  of  the  arm, 
so  that  the  robot  is  self-contained  and  only  requires  a  DC  power  supply. 

The  complete  system  is  presently  under  development.  The  new  design  incorporates  a  major  improvement 
over  the  first  generations  of  the  RCM  robot,  the  “Ball-Worm  Transmission”  recently  developed  at  the 
Hopkins  URobotics  Lab.  This  transmission  fulfills  the  need  for  implementing  a  simple  and  small  no¬ 
backlash  (no  play  between  the  input  and  output  shafts)  rotational  transmission  for  miniature  surgical 
robotics.  With  this  addition  the  RCM  exhibits  superior  motion  tracking  and  positioning  capabilities. 


66 


A  View  of  Image-Guided  Therapeutic  Procedures,  Surgical  Simulation  and  Medical  Robotics 


4.3  Localizer 

The  optical  localizing  system  (Hybrid  Polaris,  Northern  Digital,  Waterloo,  Canada)  is  used  to  determine  the 
orientation  and  position  of  tracked  objects  relative  to  the  camera  system.  Objects  are  tracked  by  rigidly 
attaching  retro-reflective  spheres  or  active  infrared  LED's  (IREDs).  The  spheres  or  IREDs  can  be  detected 
by  the  camera  system  and  used  to  determine  the  location  and  orientation  of  the  object.  This  version  of  the 
Polaris  can  track  up  to  3  active  and  3  passive  tools  simultaneously  and  is  controlled  via  the  serial  port  of  the 
host  computer. 

By  applying  reflective  spheres  directly  to  the  table,  we  are  able  to  track  the  CT  table  and  gantry.  Similarly, 
tracking  spheres  located  on  the  robot  and  end-effector  will  enable  us  to  directly  monitor  the  robot  position. 
This  is  especially  important  as  a  safety  feature  in  order  to  verify  the  robot's  own  encoders.  Finally,  a  dynamic 
reference  base  (DRB)  tracker  applied  to  the  patient  will  enable  us  to  dynamically  reference  the  patient 
compensating  for  patient  movement  or  providing  a  warning  when  motion  occurs.  The  optimum  use  for  the 
localizer  is  still  under  discussion,  but  additional  uses  include  robot  calibration  upon  start-up,  and  assistance 
in  marking  the  biopsy  entry  point. 

4.4  Biopsy  Scenario 

The  scenario  envisioned  for  robotic  spine  biopsy  is  as  follows: 

1 .  The  patient  is  positioned  on  the  table  and  a  series  of  axial  scans  are  obtained 

2.  The  scans  are  transferred  from  the  operator’s  workstation  to  the  CT  workstation  over  an  Ethernet 
connection  using  the  DICOM  protocol 

3.  The  user  interface  software  allows  the  physician  to  select  the  axial  scan  of  interest  and  the  region  to 
be  biopsied  (entry  location  and  target  point) 

4.  The  entry  location  for  the  biopsy  is  marked  on  the  patient’s  skin  (using  the  laser  lights  on  the 
scanner  and  measuring  off  the  centerline  as  necessary) 

5.  The  robot  is  manually  positioned  at  the  skin  entry  point 

6.  The  robot  automatically  orients  the  needle  and  inserts  it 

7.  A  CT  scan  is  obtained  to  verify  the  needle  position 

8.  The  biopsy  sample  is  taken 

The  testbed  will  be  verified  on  phantoms  and  cadavers  before  any  clinical  trials  are  planned.  The  initial  goal 
is  to  evaluate  the  accuracy  obtainable  using  robotically  assisted  biopsy  as  compared  to  the  current  manual 
technique  as  described  in  Section  2.  The  interventional  phantom  shown  in  Figure  5  will  be  used  in  these 
initial  studies. 


Figure  5.  Interventional  phantom 

(Courtesy  of  CIRS,  Inc.  www.cirsinc.com1 

5.  SYSTEM  SOFTWARE  DESIGN 

Current  software  systems  deployed  in  surgical  environments  do  not  lend  themselves  to  open  software 
architectures  that  utilize  off-the-shelf  (OTS)  components.  These  systems  are  developed  from  a  single 
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functional  perspective;  even  if  multiple  functional  capabilities  are  integrated,  they  are  tighdy  integrated  in  a 
closed  fashion  that  results  in  large,  cosdy,  monolithic  systems.  It  then  becomes  difficult  to  integrate  these 
systems  with  systems  dedicated  to  other  functional  areas.  An  example  of  this  is  the  need  to  offload  images 
from  the  CT  workstation  to  the  NT  workstation  in  our  testbed  environment  (see  Figure  1).  A  more 
desirable  approach  is  to  view  technolog}'  integration  via  open  software  architectures,  where  various 
hardware  and  software  components  may  be  integrated  on  top  of  a  common  software  “bus”. 

Our  approach  to  integrating  the  hardware  and  software  components  of  the  robotic  biopsy  testbed  is  to 
develop  functional  component  wrappers  for  each  component  and  integrate  them  on  top  of  an  open 
architecture.  The  “wrappers”  will  shield  implementation  details  from  other  components,  reducing 
“hardcoded”  dependencies  between  components  and  enabling  the  dynamic  composition  of  functionality  to 
meet  application  requirements.  At  the  same  time,  the  architecture  must  support  drill-down  optimizations, 
fault  tolerance,  and  error  handling,  to  ensure  the  complex  requirements  of  the  application  domain  are  met. 
To  provide  a  clearer  picture  of  how  these  components  may  interact  to  support  the  robotic  biopsy 
application,  we  describe  component  collaborations  using  the  Unified  Modeling  Language  (UML) 
collaboration  diagram  shown  in  Figure  6. 
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Figure  6.  Testbed  component  collaboration  diagram 


This  figure  shows  the  collaboration  between  high-level  software  components,  hardware  components,  and 
the  physician.  Although  UML  collaboration  diagrams  are  typically  reserved  for  software  components,  we 
find  this  a  useful  notation  for  expressing  workflows  between  hardware  and  software  components,  as  well  as 
indicating  the  role  of  operator  intervention.  Steps  are  numbered  according  to  the  sequence  in  which  they  are 
performed.  This  diagram  shows  that  the  CT  workstation  first  acquires  a  study  of  images  from  the  Mobile 
CT.  The  scans  are  then  sent  to  the  NT  workstation,  and  path  planning  is  done  to  determine  the  slice  and 
planned  path  to  the  lesion.  The  NT  workstation  tells  the  Optical  Localizer  to  track  objects  in  the  application 
space,  which  is  done  continuously  for  the  rest  of  the  procedure.  The  Mobile  CT  and  PAKY-RCM  robot  are 
then  positioned  manually  by  the  physician.  The  NT  workstation  then  orients  the  needle  by  transmitting  path 
and  coordinate  information  to  the  robot,  and  the  needle  is  driven  to  the  target.  A  verification  scan  is  then 
taken. 

The  figure  shows  at  a  high  level  the  interactions  that  must  take  place  between  components.  Each  interaction 
will  in  fact  be  realized  at  a  lower  level  as  a  number  of  messages  sent  between  objects  in  the  environment.  In 
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order  to  facilitate  the  interactions  between  components,  the  architecture  must  provide  some  common 
sendees  to  tie  the  components  together.  Such  services  include  DICOM  communication  (CT  and  NT 
workstations),  task  synchronization,  asynchronous  and  synchronous  communication,  error  handling  and 
event  notification,  logging,  coordinate  transformations,  and  redundant  verification  (between  the  localizer, 
mobile  CT,  and  robotic  device).  An  important,  domain-specific  example  is  the  mapping  of  coordinate 
systems  between  the  optical  tracker,  NT  user  interface  software,  and  the  robot.  This  integration  task 
requires  a  precise  mapping  of  the  relative  coordinate  systems  of  each  component  in  order  to  carry  out 
visualization  and  verification  tasks  in  the  application  space. 

The  current  architecture  is  being  developed  in  C++  using  DICOM  as  the  image  transfer  protocol  and 
communication  with  hardware  components  via  wrapper  libraries  that  talk  with  specific  interfaces  (such  as 
serial  ports  or  special-purpose  device  cards).  All  component  wrappers  are  being  developed  with  well-defined 
object  interfaces.  We  view  this  as  a  first  step  towards  a  more  general,  open  software  architecture  where 
hardware  and  software  components  are  integrated  and  configured  dynamically  to  meet  the  requirements  of 
families  of  surgical  applications  [16].  We  are  currently  evaluating  state-of-the-art  middleware  technologies 
such  as  CORBA,  DCOM,  and  Jini  for  this  purpose. 


6.  CONCLUSIONS 

This  paper  described  an  ongoing  project  in  developing  a  robotic  biopsy  testbed.  Once  the  system  is 
integrated,  it  will  be  used  to  compare  robotically  assisted  biopsy  to  the  current  manual  technique.  This  will 
require  the  continued  close  cooperation  between  the  engineers  and  physicians  and  the  development  of 
appropriate  measures  to  judge  the  success  of  the  procedure. 
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